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ABSTRACT 

Major conclusions of this report prepared by the 
Committee on Vocational Education Research and Development are that 
the $250 million spent by the 0. S. Office of Education (USOE) on 
vocational education research and development during the last 10 
years has not had documented, widespread impact because of lack of 
coherent policy, administration, and leadership. Chapter 1 gives an 
overview of. the report by the committee (formed in 1974), whose two 
' major tasks were to review and assess the research and development 
activities sponsored by OSOE under the Vocational Education Act of 
1963 as amended in 1968, and to recommend changes. Chapter 2 presents 
a brief discussion of vocational education in this country: how it 
differs from general education, how it has changed over the years, 
and what issues it faces today. Chapter 3 summarizes the., history of 
vocational education research and development (R8D) legislation and 
funding. Chapter 4 gives the results of the. committee's assessment of 
vocational education R&D programs which were obtained by reviewing 
large-scale evaluations, by searqhing. for examples of successful 
projects, and by interviewing people involved in the R&D program. 
Chapter 5 presents the committee's description and assessment of the 
administration of vocational education" RSD program and their 
recommendations for changes in the program's structure and management 
that are intended to result in an integrated R&D system. Details on 
procedures and findings are appended. (HD) 
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Preface 



Almost all public policy decisions— in fact, almost all decisions—must 
be made on the basis of incomplok information. In the past decade, 
federal policy makers-working in education who have tried to base deci- 
sions on program, evaluations have been hindered by the insufficient 
quality and quantity of information. Since many of the programs being 
.evaluated are planned and carried out by local school districts with only 
minimal supervision by state. and federal officials, the evaluator needs, 
but seldom can secure, data from almost every school district in the 
nation. The evaluation of vocational education research and develop- 
ment (r&d) has similar problems. Vocadonal education r&d is conduct- 
ed in literally hundreds of different places: in all states, in many large 
school districts, and at most major universities. Its evaluation is further 
complicated by the need to assess th6 utility of each of the r&d products 
in a setting appropriate for its intended use. Complete data to allow such 
an evaluation are not and probably never will be available. The Commit- 
tee on Vocational Education Research and Development was concerned 
with these problems of evaluation as it sought to recommend changes for 
the future. 

The Committee was formed in 1974 to perform two major tasks for the 
U.S. Office of Education: to review and assess the research and develop- 
^ment activities-^sponsored by the Office of Education under the authority 
of the Vocational Education' Act of 1963 as amended in 1968; and to 
recommend changes in r&d policies and programs for the coming de- 
cade. The Committee, which directed the writing of this report, was com- 
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posed of professionals from diverse parts of the vocational education 
community— universities, r&d institutions, national' organizations, and 
state and local education agencies— as well as behavioral and social sci- 
entists and a labor union or'ficial. % ^ > . 

Although we were asked to study only the r&d component of voca- 
tional education, we woutd have liked to review vocational education as 
a whole. How, after all, can one evaluate applied r&d in terms of im- 
proving programs without considering the goals of the programs it is 
designed to serve? The objective^; of some programs are clearly defined, 
but their appropriateriess has not been thoroughly investigated: some 
programs emphasize me^eting the needs of employers, while others stress 
the sometimes antagonistic goal of increasing students' employment op- 
tions. Most programs are designed to increase student adaptability to the 
existing world of work, but an increasing number attempt to provide 
students with skills and attitudes that can change t|ie work, place. After 
much soul-searching and after rereading our terms of reference, we de- 
cided to call for research on program objectives and for the convening of 
a panel of consultants similar to the panels that brought about major 
changes in vocational education in 1963 and 1968, Research on objec- 
tives should include the development of clear definitions, an examination 
of the suitability of objectives, and suggestions as to how to evaluate the 
success of programs in meeting their stated objectives. 

The Committee and its staff gathered information from a wide variety 
of sources: we read many reports and talked to people involved in voca- 
tional education and its r&d, other education, business, and labor. This 
widespread search yielded disappointing results: many people believe 
that vocational education r&d has been beneficial, but there have been 
few efforts to measure its impact objectively. Although we had difficulty 
arriving at conclusions regarding the impact bf r&d, it was relatively easy 
for us to reach conclusions about the administration of the r&d program. 
The need for important changes in the management and structure of the 
program was agreed upon.unanimously and we were able to recommend 
strategies for change. However, we were not able to recommend a solu- 
tion for every important problem. 

The Committee wanted to include in this report a recommendation on 
the coordination of various federal r&d programs of relevance to voca- 
tional education, but we were unable to find a satisfactory solution to the 
widespread problem of lack of coordination. We believe that the work of 
three particular programs should be coordinated: the vocational educa- 
tion R&D program in the Bureau of Occupational and Adult Education of ' 
the U.S. Office of Education, the education and work program at the 
National Institute of Education, and the manpower r&d program in the 
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Department of Labor. The Committee on the Department of J^abor 
Manpower R&D of the National Research Council recommended the. 
formation of a National Center for Mfanpower Study, and a similar sug- 
gestion for a National Institute of Vocational Education has been pro-' 

. posed to Congress. Neither of these suggestions proposes the necessary 
linkages with the substantial r&d programs on technical education being 
conducted by the armed forces, the Agency for International Develop- 

„ment, the Department of Agriculture, and many other public and private 
agencies. Although the Committee agreed that the directors of relevant 
R&D programs ishould share findings and products to minimize Unneces- 
sary duplication of effort, we were not able to agree on a mechanism for 
accomplishing the needed coordination in planning. 

Perhaps the Committee's greatest difficulties came in assigning relative 
priorities to different research topics. So much needs to be done. Only 
the area of counseling and guidance has received sustained support for 
the past ten years,' and even in this area all agree on the vast amount of 
work still to be accomplished. It is inevitable that a group as diverse as 
this Committee will not agree oft'every point. Most of our disagreements, 
however, were on matters of emphasis. For example, some Committee 
members, emphasizing the enormous gaps thai remain in the education 
of minorities arid women, would have recommended extensive additional 
work; others, emphasizing the progress that has been made, would have 
stressed the need to work on other topics. While we agreed to disagree on 
research priorities, we had no difficulty at all in decrying the paist prac- 
tice of emphasizing an area of research for a year and then ignoring it for 
two or three years. 

Despite our differences, we have agreed that this report can lead to 
improvement in vocational education r&d— an important means to the 
end of offering improved vocational education services to individuals 
and society. 

RUPERT N. EVANS, Chairman 

Committee on Vocational Education Research and Development 
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Highlights 




The Committee^has found that the $250 million spent by the U,S. Office 
of Education on vocational education: research and development during ^ 
the last ten years has not had documented, widespread impact. Although* 
the Comniittee did not have adequate data and models* for a rigorous 
evaluation, the available data do not indicate that ^vocational education 
research and devfelopmeiit (Rad) findings and products have had an in- 

. fluence on the knowledge; skills. oVempljxyAbility;6f large numbers of 
•students. The Committee believes that vocational education r&d shares 

- with educational R&p a 'lack of both demonstrated impact on students . 
,and methods for rigorously measuring impact. ^ - 

There are several major reasons for the limited impact of vocational 
education R&d: _ .i 

^' 

• Priorities for r&d have*been based more Ion political and bureau- 
cratic considerations than on 'the results of previous research. Further, 
priorities have changed frequently so that resbarch on any given issue 
has not been continued long^ enough to yield results. There has been 
virtually no R&d on some important problems while research on other 
problems has been continued beyond thfe point of fruitful work.^ 

• Geographic restrictions on the distributiDn of R&d funds haye re- 
'^suited in the failure of the R&d program to ad^uately address prolilems 

of national or multistate scope and, sometimt s, to fund the most qual- 
ified researchers, j 
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2 ASSESSING VOCATIONAL EDUCATION RESHARCM AND, DEVELOPMENT 

• There has been a lack of coordination of the separately adminis- 
tered parts of the federal R&D program. 

' • Systematic and effective strategies have not been implemented for 
disseminating and encouraging utilization of r&d products and results. 

• There has been no systematic elTort to maximize the impact of the 
vocational education r&d program, and the Tew', attempts to measure 

^»',inipact have met with only limited^success. The Committee believes that 
any program of applied r&d should emphasize maximizing and measur- 
ing impact. 

• Most of the early expenditures \vere not used to produce usable r&d 
products but rather principally to develop r&d institutions and to train 
R&D personnej. This occ\irred because there-were few trained researchers 
in vocatibnal education and few social scientists knowledgeable about 

. vocational education when substantial R&D funding began in 1965. 

The Committee has concluded that these deficiencies stem from a lack 
of coherent policy, administration, and leadership in the vocational edu- 
cation R&D program. 

The Committee believes that improvement in vocational education 
R&D requires, first and foremost, stable policy, leadership, and priorities. 
In this report, the Committee recommends specific ways to build a well-, 
integrated system of vocational education r&d in order^to imprOve the 
quality and impact of vocational education. The recommended plan for 
evaluation and changes in the management and structure of the r&d 
•program are necessary, if vocational education is to be continually im- 
proved by its r&d. 

THE CHANGING FOCUS OF VOCATlONAi: EDUCATION 

Since 1917 the primary objective of vocational education has been to 
prepare students for jobs in order to meet the labor market needs of the 
American economy." Beginning in the 1960s, vocational education be- 
came increasingly concerned wi-th meeting the needs of its students and 
of society. For example, vocational education now ^akes a special effort 
to ^serve economically and socially disadvantaged students and to in- 
crease the flexibility of all ^vocational education students in choosing 
careers and changing occupations. \ . ; 

Chapter 2 presents a brief discussion of vocational education in this 
country: how it differs from general education, how it has changed over 
the years, and what issues it faces to'day. 'Although vocational education 
- R&D has -touched on some of these issues, it has rarely addressed b?oad 
% philosophical questions or played a significant role in clarifying major 

' ' • . . 
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issues. Instead, r&d has most often been concerned with more localized 
problems, descriptive studies, curriculum development, and demonstra- 
tions. Research on some important and difficult issues, such as measur- 
ing the actual benefits of vocational education, is nearly nonexistent. A 
review of R&D on nine major topics related to vocational education is 
presented in Appendix A. 

The Committee believes that the primary purpose of vocational educa- 
tion R&D is the continual updating and improvement of vocational edu- 
cation programs. Vocational education r&d has been and should be ad- 
dressing questions and problems arising in the practice of vocational 
education. In addition to this supportive and passive role., however, the 
Committee believes vocation.^\l education r&d can and should contribute 
to ihe clai*ification of the purposes of vocational education and help the 
program adapt as society changes. Defining objectives, measuring the 
actual benefits of existing programs, and initialing exploration of new 
subjects in vocational education, rather than simply reacting to problems 
of existing programs, are important activities for the r&d program. Voca- 
tional education R&D should be applied r&d, as Congress suggests, but 
the term **applied" should be interpreted more broadly than in the past; 
vocational education research should include the development of re- 
search methods. (The history of vocational education r&d legislation and 
funding is summarized in Chapter 3.) 

ASSESSMENT OF VOCATIONAL EDUCATION R&D 

The Committee attempted to assess the vocational education r&d pro- 
gram by reviewing large-scale evaluations, by searching for examples of 
successful projects, especially those with objective measures of impact, 
and by interviewing people involved in the r&d program. The results of 
that assessment are found in Chapter 4. 

The Committee's assessment v/as hampered because the objectives of 
vocational education r&d have not been clearly defined. The Committee 
also had difficulty finding evidence of impact, partly because there are 
insufficient data to allow for a comprehensive evaluation of vocational 
education r&d. Impact measures have often been subjective and difficult 
to validate and have failed to measure the long-term eff'ects of r&d. 

The Commissioner of Education should ensure that usoH develop a 
comprehensive plan, for evaluation of vocational education R&D. The 
objectives of r&d should be identified in conjunction with an examina- 
tion of vocational education and its actual benefits. Evaluation criteria 
should be developed and a sample of projects should be extensively eval- 
uated. Longitudinal studies of vocational students and their employers 
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should be conducted, and those studies should include measures of stu- 
dents* job satisfaction, continuation of education, job mobility and wag- 
es, employers' satisfaction, and savings in trainmg costs. 

Because vocational education is relatively new and rapidly changing, 
the ongoing progranri of r&d should be supplemented by a panel of con- 
sultants appointed by the President and charged with studying all voca- 
tional education, including its r&d. This panel should be convened every 
five years to study vocational education and its r&d. 

ADMINISTRATION OF THE VOCATIONAL EDUCATION R&D PROGRAM 

In Chapter 5 the Committee presents its description and assessment of 
the administration of the vocational education R&D program. The Com- 
mittee recommends many changes in the program*s structure and man- 
agement that are intended to result in an integrated r&d system. These 
changes are designed to meet several goals: / 

(1) to facilitate communication and coordination among parts of the 
R&D program and to define the roles and interrelationships among insti- 
tutions involved in the program; 

(2) to ensure that long-term problems, especially those of national 
and multistats scope, are studied; 

(3) to ensure that the needs of groups such as minorities, women, the 
disadvantaged, the handicapped, and those who do not speak English as 
their first language are addressed; 

(4) to minimize political and bureaucratic influences on r&d priori- 
ties and distribution of funds; 

(5) to increase the coverage, quality, and utility of the information 
collection and retrieval system; and 

(6) to increase the extent and quality of dissemination and utilization 
of R&D results. 

These six goals are not ends in themselves, but are means that, in the 
Committee's judgment, will lead to improved vocational education r&d 
and service to students. The Committee's major recommendations on the 
administration of the r&d program are summarized here. 

Consolidation of Parts of the Vocational Education R&D Program Con- 
gress should consolidate research, development, and demonstration 
(Parts C, D. and I of the 1968 Vocational Education Amendments) in 
new legislation and in the structure of the U.S. Office of Education. 
Research should receive at least 20 percent of the funds appropriated. 
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Commissioner's Share of Funcb National or multistate problems should 
be addressed by the Commissioner's share of funds (50 percent) while 
states should continue to address state and local problems. The current 
geographic limitation on awards of the Commissioner's share should be 
removed. 

Procedures for Setting Priorities The Commissioner of Education and 
state directors of vocational education should initiate a rigorous system 
of setting priorities, using continuing advisory groups and management 
information systems to determine long-range plans forR<s.D. More input 
from researchers should be considered in establishing priorities. 

National Vocational Education R&D Centers The Congress and the Com- 
missioner of Education should ensure the continued existence of at least 
one national vocational education r&d center, which should receive 
long-term support foriiddressing national and regional problems, includ- 
ing those identified by the center(s). The center(s) should coordinate 
their work with the research activity within the states and should assist in 
disseminating research products and training r&d personnel. 

Information Collection and Retrieval The Department of Health, Educa- 
tion, and Welfare (hew) should support a comprehensive and well-inte- 
grated information resource system lirked to a dissemination network 
serving practitioners. A clearinghouse for vocational and technical edu- 
cation should include the abstracting and indexing functions of Ab- 
stracts of Instructional and Research Materials (aim/arm) and be well 
coordinated with other vocational education r&d activities. Every voca- 
tional education r&d project should be required to submit final reports 
10 Educational Resources Information Center (eric) and aim/arm. Pro- 
vision needs to be made for collecting and disseminating audio-visual 
materials. 

Information Analysis The Commissioner of Education should establish 
an information analysis program to transform R<<^n information on crit- 
ical problems into appropriate forms for diverse user groups. This pro- 
gram should provide interpretations useful to each of those groups of. 
users. Effective dissemination techniques should be developed and im- 
plemented for these products. 

Utilization A significant proportion of federal R&D funds should be des- 
ignated for dissemination and utilization, under the direct responsibility 
of the Bureau of Occupational and Adult Education of usoe. In conjunc- 
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lion with the information resource system for vocational education, a 
comprehensive dissemination and utilization plan involving the national 
R&D center(s), research coordinating units, state and local education 
agencies, and other organizations should be developed. User training 
programs should be conducted to improve the flow of information from 
the resource system to practitioners. ^ 



2 



The Changing 
Focus of 
Vocational 
Education 



In a community college in Michigan, there are two lunch' lines in the 
cafeteria: one serves the regular $0.75 lunch; the other, for $1.25, serves a 
gourmet meal prepared by students in the chef training program. In a 
high school in Illinois, handicapped and other vocational students oper- 
ate a combination motel, restaurant, and gas station completely on their 
own. In another high school district, vocational students have built and 
sold' 35 residential houses in the last 35 years, combining the efforts of 
students in drafting, interior decorating, sheet metal shop, and building 
trades. These' are only three examples of scores of ways in which voca- 
tional educators are trying to meet their primary, objective: preparing 

' students for work. 

Vocational education is unique in a number of ways. It treats students 
as practitioners, preparing them for skilled entr>'-leyel jobs not requiring 
a baccalaureate. It offers training in specialize^ skills such as nursing, 
auto mechanics, and irrigation technology, usu-Uy in high schools or in 
two-year post-secondary schools. It frequently iv es instructional settings 
other than (and in addition to) the traditional ci ssroom. Its success in 
terms of placing students in jobs is highly dependLi.n upon the economy. 
In high schools it serves primarily students of lower Hcioeconomic status 

• and less academic ability than the general student pc^mlation. 

This chapter sets vocational education in its sociai.and historical con- 
text and presents the .issues that ha\'e faced vocatio nal educators and 
researchers. These issues are also discussed as they relate to present and 
future vocational education and its R&D. / 
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8 ASSESSING VCXTATIONAL EDUCATION RESEARCH AND DEVELOPMENT 

BACKGROUND 

The notion that schools should prepare students for work is relatively 
new. Schools began to introduce practical subjects, which were designed 
to "train the hand as well as the mind," toward the end of the nineteenth 
century, when most young people were taking advantage of free public 
elementary education, and an increasing number were continuing 
through high school. Manual training, domestic science (home econom- 
ics), agriculture, and business courses attracted many students, and en- 
rollment in these curricula grew rapidly. Around the turn of the century, 
however, some of the manual training educators and many agricultural^ 
business, and industrial leaders became aware of the need to prepare 
people for more specific occupations. The idea of developing vocational 
education courses for high school students who wanted to prepare for 
work grew from this need. ^ 

Three pieces of legislation were critical in the development of voca- 
tional education: the Smith-Hughes Act of 19 17 (P.L. 64-347), fhe Voca- 
tional Education Act of 1963 (P.L. 88-210), and the 1968 Amendments 
(P.L. 90-576), to the 1963 Act. The Smith-Hughes Act pr6vided..federal 
aid for vocational education offered by public secondary schools; this 
significantly increased the number of programs and students, from 
160,000 students in 1918 to almost four million in 1960. Many of these 
students were adults who wanted better jobs. The Vocational Education 
Act of 1963 authorized increased funding for vocational education and 
also encouraged vocational education to shift to broader goals related to 
the development of human potential and long-term employment. Voca- 
tional-education had been first offered in public schools in the UnTted 
States during a period of thriving industrialism, when the needs of work- 
ers were considered to be secondary to the needs of the economy. At that 
time, vocational education accepted the short-term goals of meeting the 
rapidly changing demands of the labor market; it was less concerned 
with gradual trends in the labor market or with human needs. With the 
passage of the 1963 Act, the objectives of vocational education became 
much broader than thej^ had been 60 years earlier. Because of the broad- 
ened focus and increased funding, enrollment in vocational education 
programs more than doubled in the next four years. 

The 1968 Vocational Education Amendments continued the emphasis 
on serving the needs of students. An important objective of the Amend- 
ments was to provide better service to the disadvantaged, the handi- 
cappe<J- p-^l-secondary, and adult students. The legislation also provid- 
ed support for t^ career education movement by stressing career plan- 
ning as well as employment preparation. of students.. 
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TTie Changing Focus of Vocational Education' 9 
ISSUES IN VOCATIONAL EDUCATION 

Vocational education of the 1970s differs markedly from the program 
initiated in 1917. In 1974, there were 13.5 million students in vocational 
education programs, including the disadvantaged, the handicapped, and 
those needing more advanced specialized training in post-secondary in- 
stitutions. Vocational education programs are today more varied in con- 
tent, employ more sophisticated instructional methods, and attempt to 
teach much more than occupational skills. Vocational counseling plays 
an increasingly important role in vocational education, so that students 
can be better equipped to make decisions regarding occupational choice, 

The ..objectives of vocational education have been expanded since 
1917. While a major objective is still to prepare people for work in order 
to meet the heeds of the economy, a second objective, which emerged in 
the 1960s, is to increase the employment options available to each per- 
son. Vocational education has become concerned with developing flexi- 
ble occupational and decision-making skills so that students may choose 
any of several occupations after graduation. A third, usually implicit, 
objective is motivating students to learn basic academic skills. 

Researchers" have done and continue to do investigative and develop- 
mental work to improve vocational education. (The Coinmittee's review 
of R&D related to vocational education is presented in Appendix A.) 
Three particular lines of investigation have been pursued— characteris-. 
tics of students, instruction of students, and the relation of vocational 
education to work. First, vocational education researchers have tried to 
identify the characteristics of students being served by the programs so 
that the needs of the students can be^cc/nsidered when designing pro- 
grams. Vocational educators and researchers are sensitive to the need to 
serve all students, including women, minorities, the disadvantaged, and 
the handicapped. Second, vocational educators are concerned with the 
instruction r rceived by students. Accordingly, research and development 
is carried o.n to make curricula flexible, to develop new programs in 
career education, and to compare the 'effectiveness of various modes of 
instruction. Third, vocational educators are concerned with the objec- 
tives of vocational education and the relationship between programs and 
the work place. Researchers use labortmarket information to adapt pro- 
grams to the demands of the economy. They study career development 
and guidance to increase the flexibility and decision-making skill of each 
student. They also evaluate vocational education to measure the extent 
to which those programs are meeting their stated objectives and to judge 
the appropriateness of the objectives. 
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CHARACTERISTICS OF VOCATIONAL EDUCATION STUDENTS 

• Tailoring Programs to the Needs of Students 

In light of.the expanded objectives of vocational education, practitioners 
now generally believe that the characteristics ^and needs of students 
should be understood before programs can be developed to best meet 
students' needs. Several studies have shown that, compared with other 
students, vocational education students usually come from families, of 
lower socioeconomic status and have less academic, especially verbal, 
ability. Their parents usually have received less education. It is not 
known, however, why students of lower socioeconomic status, lower oc- 
cupational aspirations, or less academic ability are in vocationareduca- 
tiori programs. Researchers have i:ot determined to what extent studen^ts 
with these characteristics actively choose vocational programs over other 
programs and to what extent they are assigned b^ school administrators 
to vocational programs more often than other students. There seems to 
be a tacit assumption that these students are better prepared for work by 
vocational than general education since they learn skills for blue-collar 
jobs, which they are likely to hold because of their socioeconomic back- 
grounds. The implication that vocational education thus perpetuates so- , 
cial class distinctions has not been investigated. 

It is also assumed that vocational students learn basic educational 
skills (the three Rs) better if they are . enrolled in both vocational and 
general education courses because they are more interested in the con- 
text in which these skills are taught. Frequently, students who are ready 
to drop out of general high school programs are reportedly "turned on" 
by vocational programs and become enthusiastic students and pro- 
ductive workers. This assumption has not been questioned in a scientific 
manner. 

Equal Access to Programs 

Educators are now required to provide equal program opportunities to 
all students. Emphasis has been placed on minorities, the handicapped, 
the. disadvantaged, adults, and more recently, women and those who do 
not speak English as their first language. Vocational education has been 
encouraged to eliminate sex stereotyping in programs and to provide- 
equal access to programs associated with occupations that have been 
traditionally dominated by one sex. It is not clear to what^'extent this 
requirement is being met. There is some question as to how vocational 
education can best serve tho^c students who have been socially and eco- - 
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nomically disadvantaged. For example,, how can vocational education 
take into account differences in the cultural values, especially those con- 
cerning work, of different ethnic groups an(f provide training to maxi- 
mize students' chances of getting and holding, jobs? Rarely do teachers 
receive special training to help them meet the unique needs of different ' 
groups of students. (Suggestions for further research on meeting the 
needs of minority group members and women are contained in articles 
by Hamilton [1975] and Roby [1976].) 

INSTRUCTION OF VOCATIONAL EDUCATION STUDENTS 

Flexibility in Curricula 

The training needed by vocational education students has changed 
markedly over the years. With rapidly changing economic and techno- 
logical developments in business and industry, new skills are required 
and old skills become obsolete. Although substantial efforts have been 
made in r&d to develop a means for keeping curricula up-to-date and 
responsive to the changing needs of students, this remains a major prob- 
lem in vocational education. Institutional rigidity has sometimes resulted 
in the training of outmoded skills on obsolete equipment by teachers 
whose knowledge of industry is not current. 

Further, it is becoming increasingly likely that people will change oc- 
cupations at least once in their lifetimes, so that vocational educalion 
programs should teach multiple and generalizable skills that will prepare _ 
people better for mid-career changes. The objective of training students 
for occupational versatility has not been easy to meet. Vocational educa- 
tion Rc&D has not successfully solved the problem of training people for 
the specialized technical skills required by certain occupations and, si- 
multaneously, preparing Ihem for a broader range of job opportunities. 

Career Education 

Career education, a major r&d topic funded under the 1968 Amend- 
ments, has attempted to expand the boundaries of traditional vocational ' 
education. The essential concept of career education is that all students, 
not just vocational education students, should be exposed to career de- 
velopment opportunities throughout their school years and that every 
student should leave school with the skills necessary for job entry, wheth- 
er that student completes the tenth grade or a four-year college course. 
Career education ideally exposes students to the full range of career 
opportunities, helps them decide their occupational futures, and provides 
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education and training appropriate to their career choices. Career educa- 
tion includes vocational education in that^ occupational training for spec- 
ific skills is one of its essential elements, but it is also concerned with 
preparing all students to make career decisions and changes throughout 
their lives. 

Despite the heavy investment of r&d funds in career education pro- 
grams, research has neither established an empirical or theoretical basis 
for career education nor evaluated its effectiveness. It has not been deter- 
mined at what point in a student^s education knowledge about jobs and 
careers should be introduced, when specialized skills should be taught/or 
how career education can be individualized for students with differing 
needs and ambitions. 

On-the-job Training versus Classroom Training 

Another critical issue in vocational education instruction has been the 
identification of the advantages of classroom instruction relative to those 
of on-the-job training (which can take place under the auspices of the 
employer or in conjunction with school-sponsored cooperative educa- 
tion). Some have argued that training at the work site is the most effec- 
tive and relevant method of job instruction. In addition, if only a few 
students desire instruction in a given field, classroom training may not be 
feasible. On the other hand, classroom instruction has other advantages: 
job simulation minimizes penalties for error and allows students to learn 
at a flexible pace. The need to measure the advantages of on-the-job 
training relative to those of classroom training remains a challenge to 
vocational education r&d, 

VOCATIONAL EDUCATION AND WORK 

Objectives of Vocational Education 

• The primary objective of vocational education programs has been to 
prepare students for occupations, with a recent emphasis on equality of 
access to programs. Some other objectives, benefits, and purposes of 
vocational education have been identified but not fully studied. Voca- 
tional education programs usually serve the perceived needs of students 
and workers as well as those of employers. Success is usually measured 
by the proportion of graduates who get jobs. The costs and benefits pf 
vocational education and the distribution of these costs and benefits \q 
the public and private sectors have not been investigated. Similarly, the 
distribution of^ costs and benefits to employees and employers has not 
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/ been studied. For example/training aviation mechanics in public schools 
i reduces training costs for the private airline companies. (In theory, the 
/ more the training is specifically tailored to actual job requirements, the 
/ greater the benefit to the empiloyer.) To the extent that competition exists 
/ . among employers, the ^saving in private training costs from vocational 
/ education programs may be passed on to the consumers. The extent to 
/ which public vocational schools augment private profits or cause lower 
pri'ces has not been measujgd. 

Vocational education can:serve both public and private purposes. An 
example of a public purpose for vocational education would be helping 
people become responsible citizens by helping them to become responsi- 
ble workers. Training for citizenship, of course, is one of the principal 
purposes of schools in general. Vocational schools may have a particu- 
larly important role to play because there is evidence that people aire 
more inclined to participate responsibly in democratic government if 
they have the experience of responsible participation at work. 

Democratic responsibility in the work place has been a major political 
issue in other industrial countries for years. In October 1975, the U.S. 
Congress established the National Center for Productivity and Quality of 
' Working Life, to promote, among other things, inquiry into ihe possible 
benefits of increased participation by employees in decision making in 
the work place. In order to enhance their ability to participate responsi- 
bly at work, students in public vocational programs might be given op- 
portunities to learn, for example, what a corporation is and what a pro- 
ducers* cooperative is, how collective bargaining works, what the stock 
market does, and how time-and-motion studies are performed. They 
might also be trained in the skills required to own and operate their own 
businesses. Researchers should seek ways in which vocational education 
can help students achieve the level of economic literacy necessary to 
exercise their full rights and responsibilities at work. 

Job Placement ^ . 

Because job training has been a major goal of vocational education, 
there have been many attempts to maximize the prpportion orvocational 
education graduates placed in jobs. Two complementary strategies have 
been used— labor market forecasting and career guidance. 

Labor market information is used by vocational educators to predict 
future demand for certaiii occupations and to adapt programs to meet 
the demands. Many states have developed and are using their own labor 
market management information systems, but present forecasting meth- 
ods can be improved in at least three ways. First, labor market demand 
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forecasts should attempt to take intd account the extent to which wages 
and working conditions will change if more or fewer workers are trained. 
Second, program planners and researchers should recognize that labor 
market information from various vocational education districts must be 
coordinated in some way because workers move from place to place. 
Third, since macroeconomic or institutionally oriented employment poli- 
cy-affects total demand for labor, it should be taken into account in labor 
mai^ket forecasts. ' \ 

, Career development and guidance strive to meet the needs of individu- 
iil students while they are enrolled in programs and after graduation. The 
overall goal of career guidance is to improve the ability of students! to 
make career decisions. Traditionally, guidance counselors waited until 
students came to them with career or job problems. Now some counsel- 
ors are trying to anticipate and present problems by providing students 
with career information and decision-making skills that will reduce the 
frequency of decisions based on incomplete information or inappropriate 
choice strategies. ^ ^ 

Educational personnel are increasingly concerned with providing stu- 
dents more information on' available instructions and training programs. 
Students might find it helpful to learn about the experiences of graduates 
and dropouts from these institutions. At present, there is more informa-^, 
tion available to help institutions select individuals than to help individu- 
als select institutions. The e^i|cts on students and institutions of the 
availability of this information is'hot known. \ 

Evafuation of Vocational Education Programs 

A necessary step in improving the connection between vocational'educa- 
tion and work is evaluation, which measure^ the extent to which prcn 
grams are meeting their stated objectives. Past evaluations of vocational! 
education programs have rarely been adequate. The studies most ofteri| 
cited as model evaluations generally- need improverrient. Larger and bet-i 
ter-designed samples, more appropriate experimental and questionnaire^, 
design, better measurenient of criterion and background variables, and\ 
more suitable statistical techniques could strengthen future evaluations, i 

Further, some believe that current evaluations offer little assistance to 
the vocational education policy maker, curriculuni developer, or teacher 
because the criterion measures inadequately reflect .program success: 
evaluators have most often measured success in terms of initial job-place- 
ment, which is determined by many factors, including the availability of 
jobs, the social status, personality, and intellect of the student, and 
chance. The existing criterion variables could be supplemented with new 
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measures of other aspects oPprogram success, for example, job satisfac- 
tion, job turnover rates, the socioeconomic mix of students, and changes 
in student self-perceptions. ^ 

CONCLUSIONS I 

The issues facing vocational education ^also affect other behavioral and 
social sciences.. In the past, vocational education researchers have been 
able to draw on the work of social and behavioral scientists in areas such 
as'human learning and development. However, in some cases, vocational 
education r&d has proceeded without the benefit of established social 
and behavioral science theories or an extensive knowledge base. The 
Committee' hopes the education -and manpower work program of the 
National Institute of Education and ^he research program of the Depart- 
ment of Labor will provide support for the developirient of coherent 
theoretical perspectives leading to more useful applied research in voca- 
tional education. 

In addition, to the exchange of knowledge and theories on substantive 
issues, methodological advances made by other social science researchers 
can benefit vocational education researchers. However, existing R&D 
methods have sometimes been inadequate or inappropriate for use in 
vocatioiial education. In order to|meet the methodological needs of vo- 
cational education R&D, the USOE definition of applied research should be 
expanded to include the developriient of research tools. 

In the past 60 years, vocational education has broadened its objectives 
in response to changes in American society. Enrollment in programs has 
expanded t© include groups of students never before served by vocation- 
al education. Diverse needs of students have been addressed as societal 
pressures demanded. In the future, yocatiqnal educators and researchers 
may want to take a' more active role, anticipating altered demands of the 
labor market and of society. , / 

Most of the past research in vocational education cannot be general- 
ized beyond the immediate situation that was studied. Questions with 
' far-reaching implications have not been investigated; for example, alter- 
native instructional settings have not been successfully compared, highly 
flexible and generalizable curricula have not been developed, ^nd the 
objectives of vocational education have not been carefully examined. If 
research is to improve the /education of vocational students, it must be 
more far-sighted, expanded' in scope, and improved in quality. 
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HISTORICAL BACKGROUND OF EDUCATIONAL RESEARCH 

FEDERAL COMMITMENT TO EDUCATIONAL R&D 

A national comniitment to educational R&D began a half century before 
a commitment to vocational education R&D. The federal Department of 
Education, established in 1867, was charged with data gathering and 
dissemination of statistics that would "sho\y the condition and progress 
of'education in .the* several. States and Territories .. . " (U.S. Depart- 
ment of HEW 1969, p. 47). Later; these duties were assumed by the U.S.^ 
Office of Education and until the mid-1950s they, constituted the total ^ 
federal commitment to educational r&d. (Clark 1974, p. 4). 

In 1953, the new Commissioner of Education, Samuel Brownell (1955, 
p. 2), sought to expand the i^&d functions of the U.S. Office of Education 
(usoE): "If 1 were asked to name the one field in which the Office can be 
of greatest service at this time, 1 should answer ^educational research'". 
In 1954 the Cooperative Research Act (P.L. 531) authorized the Com- 
missioner of Education to enter into "contracts or jointly financed coop- 
erative, arrangements with universities and colleges and state educational 
agencies for the conduct of research, surveys, and demonstrations in the 
field of education." Although the Act was not funded until 1956, and 
then at a low level, it represented the first federal commitment ^to an 
educational R&D role broader than data collection and disseinination. 
' The Elementary and Secondary Education Act of 1965 expanded the. 
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usoE involvement in educational R&D. A wide variety of new programs 
were initiated under the authority of Titles III and lY of the 1965 Act. In 
an effort to improve dissemination and utifization of research results, 21 f 
RAD centers were establfshed, as well as 20 regional laboratories, to de- \ 
velop projects and interact directly with school systems. To complement 
its investments in educational R&D, usoE funded the Educational Re- 
sources Information Center (eric) in 1966 to ser\'e as an information 
collection and retrieval system for providing ready access to educationaj^^ 
literature. By the early 1970s, "literally thousands of TJitle III projects 
were funded in local school districts and the federal investment in educ^-, 
tion R&D (loosely defined) was approaching, $200 million (per year)" 
(Clark 1974). 

Perhaps as a result of the rapid growth of the federal mvestmcnt.m 
educational r&d, support grew for the establishment of the National 
Institute of Education (nie) to serve the educational community in much 
the same manner as the health community is served by the National 
Institutes of Health. The Educational Amendments Act of 1972 autho- 
rized the formation of NiE toi ^help to solve orto alleviate the problems 
of, and promote the reform and-renewal of, American education; ad- 
vance the practice of education, as an art, science, and profession; 
strengthen the scientific and technological foundations oCeducation; and 
build an effective educational research and development system" (U.S.. 
Department of HEW 1973, pp. 9-10). 

FEDERAL COMMITMENT TO VOCATIONAL EDUCATION R&D • 

The Smith-Hughes Act of 1917 and the George Barden Act of 1946, 
which provided federal funds foi^ vocational education, permitted sup- 
port of studies and reports designed to improve the administration and 
management of vocational education programs, but neither provided 
specifically for r&d funding.-Prompted by the report of the 1961 Panel of 
Consuhants on vocational education (U.S. Department of hew 1963), 
the Vocational Education Act of 1963 explicitly provided substantial 
funding Tor vocationaleducation r&d. The Panel observed that most of- 
the small amount of r&d in vocational education had been' applied re- 
search; almost none fed been^:$^rimental research under controlled 
conditions. Most of the research had been local in scope and was little 
known outside its own locality. The Panel also noted that a considerable 
amount of the research had been devoted to the collection of data, with 
little attention to interpretations of the data collected and their implica- 
tions. " , v J ♦ 
In its report, the Panel noted several factors that had contributed to 
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some of the shortcomings of vocational education research. First, few 
individuals had been trained for research in vocational education, and 
vocational educators did research only to solve problems, not to prevent 
them. Second, much of the research in vocational education was done to 
fulfill requirements for graduate degrees; these requirements fostered 
minor studies rather than comprehensive research projects. Third, com- 
prehensive research is facilitated by adequate financing and organiza- 
tional structures, neither of which was available. (Other reasons for the 
absence of'-major research efforts can be found in the full report of the 
Panel.) 

The research committee of the 1961 Panel recommended funds ear- 
marked for research and advocated the establishment of a research cen- 
ter or clearinghouse to coordinate^stimulate, and conduct r&d activities. 
^Section 4(c) of the Vocational Education Act of 1963 reflected the 
Panel's concern with research: 

Ten per centum of the sums appropriated pursuant to section. 2 for each fiscal 
year shall be used by the Commissioner to make grants to colleges and universi- 
ties, and other public or nonprofit private agencies and institutions, to State 
boards, and with the approval of the appropriate State board, to local educa- 
tional agencies, to pay part of the cost of research and training programs and of 
experimental, developmental, or pilot programs developed by such j 
- boardsr'or agencies, and design'e^^ the sf>ecial vocational education 

needs of youths, particularly youths in economically depressed communities 
who have academic, socio-economic, or other handicapsWat prevent them from 
succeeding in the regular vocational education, programs. 

The 1963 Act also provided for the creation of the ad hoc Advisory 
Council on Vocational Education (the Essex Council), which reviewed 
the entire vocational educational program between 1963 and 1967. In 
December 1967 this group presented a series of recommendations that 
expressed dissatisfactions with the nature of the research and the admin- 
istration of the research program. The Essex Council recommended re- 
ducing administrative complexities, providing specific trainijpg for the 
handicapped, authorizing work-study programis and-residential vocation- 
al schools, and increasing the emphasis given to post-secondary and 
adult programs. The Council recommended changes in federal-state rela- 
tionships in order to give greater latitude to the states in both program 
planning and management. It was recommended that ten percent the 
funds appropriated for vocatiorral education continue to be available for 
research-related' activities, including the support of state research coordi- " 
nating units and state research programs. The Council also recommend- 
ed that funds be allocated directly by the Commissioner of Education to 
support research on critical problerns of national scope. 
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Like the 1961 Panel, the 1967 Essex Council wanted the administra- 
tion, supervision, and coordination of the research funds to remain in the 
vocational education division of the U.S. Office of Education in order 
that such research might bear directly upon the needs of vocational edu- 
cation. 

The 1968 Amendments incoqDorated many of the Essex Council's rec- 
ommendations, including continuation of the ten percent authorization 
for research and a new division of research funds on a 50/50 basis be- 
tween the U.S. Commissioner of Education and the states. The Amend- 
ments specified that, in addition to research, exemplary projects and 
curriculum development be supported with r&d funds. 

OVERVIEW OF VOCATIONAL EDUCATION R&D 

Between 1965 and 1971 (inclusive), the U.S. Office of Education spent 
close to $250 million lo support almost 5,000 vocational education re- 
search and development project's. This section gives a brief description of 
the funding, priority areas, and general characteristics of that r&d, which 
was conducted under the 1963 Act and 1968 Amendments and adminis- 
tered by USOE. Project listings, abstracts, and files were supplied by the 
— Division-of-Research-and-Demonstration-BureaU'Of-Occupalional-and — 
Adult Education, U.S. Office of Education. An evaluation of demonstra- 
tion projects was also reviewed (Development Associates 1975). Detailed 
tables summarizing the information on which this discussionjs based are 
presented in Appendix C. 

SqcUou 4(c) of the 1963 Act provided support for vocational education 
R&D for fiscal 1965 through fiscal 1969. Beginning in fiscal 1970, r&d has 
been supported under the authority of Parts C, D, and I of the 1968 
Amendments: Part C funds support research, demonstration, and curric- 
ulum development; Part D funds are used to demonstrate innovative 
vocational education or career education programs in school settings; 
and Part I funds are authorized to support curriculum development and 
dissemination. Since all funding for r&d supported under Section 4(c) 
was superseded by Part C, Section 4(c) expenditures are analyzed in 
conjunction with the discussion of Part C. - - 

Table 1 presents fiscal data for the separate parts of the vocational 
education r&d program from fiscal 1965 through fiscal 1975. The admin- 
istration requested the full authorization for any part only in fiscal 1965 
and fiscal 1966. In each year except fiscal 1971, the request for Part C has 
been for less than one-third of the authorization; the request for Part D, 
less than one-fourth; and for Part I, less than one-half 

In fiscal 1965, 1966, and 1972 through 1975 (except for Part I in fiscal 
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TABLE 1 Funds (in millions) for Parts C, D, & I and Section 4 (c) . 



Fiscal Year Authorized Requested Appropriated 

SECTION 4 (c) - 

1965 $ 11.8 $11.8 $11.8 

1966 17.8 17.8 17.8 

1967 *22.5 17.8 10.0 

1968 22.5 17.1 13.6 

1969 35.5 15.7 11.6 
TOTAL $110.1 $86.2 $64.8 

PARTC 

1969, $ 35.0. $ 0 $ 0 

1970 56.0 I.l 1.1 

1971 67.5 • * 35.8 

1972 67.5 t 18.0 

1973 67.5 18.0 18.0 

1974 67.5 18.0 18.0 

1975 67.5 18.0 18.0 
TOTAL $428.5 $108.9 



PARTD 



1969 


$ J5.0 


$ 0 


S 0 


1970 


57.5 


13.0 


13.0 


1971 


75.0 


* 


16.0 


1972 


75.0 


t 


16.0 


1973 


75.0 


16.0 


16.0 


1974 


75.0 


16.0 


16.0 


1975 


75.0 


16.0 


16.0 


TOTAL 


$447.5 




$93.0 






PARTI 




1969 


S 7.0 ^ 


SO 


$ >0 


1970 


10.0 ^ 


2.0 


.9 


1971 


10.0 


* 


4.0 


1972 


10.0 


t 


4.0 


1973 


10.0 


6.0 


4.0 


1974 


10.0 


4.0 


4.0 


1975 


10.0 ' 


4.0- 


I.O 


TOTAL 


S67.0 




$17.9 



NOTE: During each year of Part C funding, a small number of special projects of 
national scopl' have been supported by a skim-off of tht^ state allntments of the Com- 
missioner's V'lvi C funds. Funding for these projects are included in this table but in 
no others in this report. f. 

'Combined requests for Parts C. D. and I was S25.7 million; the combined appropri- 
ation was $5518 million. 

fCombincd request for Parts C, D, and I was $36.0 million: the conibined appropr?- 
tttion was $38.0 million. 
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1975), appropriations equaled requests; in fiscal 1971 and 1972 appropri- 
ations exceeded requests. In all, $284,430,000 was appropriated for voca- 
tional education Ri&D between fiscal 1965 and fiscal 1975 (of which al- 
most $250,000,000 had been spent through fiscari974); this represented 
less than 28 percent of the amount authorized. 

Appropriations for Section 4(c) ranged between $10 million and nearly 
$18 million. Part C has received $18 million per year since fiscal 1972. 
-Since fiscal 1971, Part D has received $16 million per year. Part I, the 
smallest of the three programs, received $4 million per year between 
fiscal 1971 and fiscal 1974 and was reduced to $1 million in fiscal 1975. 
However, since Part C funds can be used for any Part I activity and Part 
D can support demonstrations of new curricula, the severely reduced 
funding for Part I did not necessarily lead to a parallel reduction in 
curriculum development projects. 

PARTS OF THE R&D PROGRAM 

Section 4(c) and Part C 

Funds from Section 4(c) and Part C have been used to support grants 
and contracts for research; training programs to familiarize personnel 
with research results and products; developmental, experimental, or pilot 
programs intended to meet the special vocational needs of youth, espe- 
cially disadvantaged youth; demonstration and dissemination projects; 
and state rcus, which administer and sometimes conduct state research 
and development programs. As noted above, all of the activities specified 
under Parts D and I can also be conducted under Part C. 

While Section 4(c) funds were reserved for the Commissioner of Edu- 
cation for direct federal grants and contracts. Part C funds are divided 
equally between the Commissioner and the states. Both the Commission- 
er's share and the states' share of Part C funds are allocated to states 
according to a population and income formula. Income is a negative 
factor in the equation; that is, states with a higher per capita income 
receive less money. Different age groups in the states' populations are 
differentially weighted: ages 15~19 most heavily, ages 20-24 less, and 
ages 26-65 least. 

Federally administered Part C projects, typically 18 months long, are 
selected from proposals made i:.- response to announcements published 
periodically jn the Federal Register, Th^ states' share of Part C is used to 
pay part of the costs of r&d programs and projects in accordance with 
the state plan. Because both federal and state shares are distributed by 
state according to a population formula, and because in some states only 
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the State education agency (sea) applies for the federal share, seas some- 
times receive both shares, ' Except for a few large projects, awards for 
study of national scope are charged against the federal share allocated to 
the state in. which the national study has its headquarters. 

Funding and Project Size Under Section 4(c), projects averaged be- 
tween $60,000 and $90,000 each. In fiscal 1971, eight large career educa- 
tion models (averaging over $1,000,000 each) were funded from the 
Commissioner's share of Part C, inflating the mean project size to over 
$500,000, In fiscal 1972 and fiscal 1973, more projects were funded— at 
least one in each state — and, consequently, the mean project size , de- 
creased. With full competition in fiscal 1974, mean project size decreased 
further. From fiscal 1971 through fiscal 1975V there were between 400 and 
700 state Part C projects annually at a mean cost of $12,000 to $20,000 
each. 

Types of Projects Under Section 4(c), about 32 percent of the funding 
supported applied or fundamental research, while about 37 percent sup- 
ported experimental, developrnental, or pilot projects. In fiscal 1971, 
about 68 percent of federal Part C funds supported experimental, devel- 
opmental, or pilot projects, and in fiscal *' 1972 and fiscal 1973, all of the 
federal Part C money went to such projects. In fiscal 1974, however, only 
31 percent of Part C funds supported experimental, developrnental, or 
pilot projects and 68 percent supported research. 

For state Part jC projects, a rough classification by project titles indi- 
cates that less than 40 percent of the classifiable projects can be called 
research even in the broadest sense of the term. About. 35 percent of the 
awards went to developmental projects, and the other awards supported 
demonstrations, evaluations, dissemination, or research coordinating 
unit activities. This classification must be taken as only a rough approxi- 
mation: it is very difficult and sometimes impossible to'classify projects 
by title only, and only titles were available. Between 19 and 25 percent 
each year could not be classified at all. It seems* clear, however, that the 
world "experimental" as it is used in the phrase experimental, develop- 
mental, or pilot projects rarely refers to scientific experiments; rather, it 
usually ^eems to mean a trial accompanied by subjective evaluation. 

R&D Topics Priority areas are announced yearly by usOE for Section 
4(c) and for the Commissioner's share of Part C, They have changed . 
substantially from year to year for the 10-year period surveyed: 
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FISCAL 1965-FISCAI. 196/ 

Program evaluation 

Curriculum experimentation 

Personal and social significance of work 

Personnel recruitment and development 

Program organization and administration 

Adult and continuing education 

Occupational information and career choice 

FISCAL 5968-FISCAL 1969 

Application of manpower data to occupational education 

The student and his environment 

State and local planning techniques 

Instructional systems development 

Career development, guidance, placement, and follow-up 

Instructional facilities 

Organization and administration. 

Research and development centers 

Evaluation 

FISCAL 1971 

Same as fiscal 1968 and fiscal 1969, excluding r&d Centers 

FISCAL 1972'-FISCAL 1973 

Career education with a strong guidance and counseling component 

FISCAL 1974 
Curriculum studies 

Disadvantaged, handicapped and minorities 
Alternative work experience programs 
Guidance, counseling, placement and student follow-up 
Manpower information and systemTor education 

FISCAL 1975 

Administration of vocational education at the state level 
Administration of vocational education at the local level 
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Comprehensive systems of guidance, counseling, placement, 

and follow-through 
Services 

Educational personnel serving the educationally disadvantaged, 

handicapped, and minorities 
Curriculum, demonstration, and installation studies 

The priority areas are very broad, indicating that usoE has chosen to 
study many different topics related to vocational education rather than 
focusing on a few. (Information on the priority-setting process^and iTie 
resultant instability in priorities is presented in Chapter 5. Additional 
information on the shifts in emphasis among the priorities is presented in 
Appendix C.) 

The Section 4(c) program can be characterised 'as covering five major 
topics: (1) identification of current and emerging training needs, (2) vo- 
cational curriculum development, (3) problems of vocational education 
resource development, (4) vocational guidance and career choice proces- 
ses, and (5) adult and continuing education. In its early years, the pro^ 
gram focused most sharply on curriculum development; in fiscal 1967 
through fiscal 1969, it became involved in the development of a program 
called Educational System for the '70s (es '70). This program was intend- 
ed to focus primarily on equipping each high school student with a spec- 
ific marketable skill plus the academic prerequisites to enter a two- or 
four-year post-secondary institution. Throughout the 4(c) program, sup- 
port was also given to the establishment of new institutional, programs, 
including funding teacher-administrator in-service training institutes, na- 
tional vocational education r&d centers, and state research coordinating 
units. 

Over the period fiscal 1971 to fiscal 1975, by far the largest portion of 
the Commissioner's share of Part C funds was spent on career education 
and career guidance. This is also the only topic that received funds every 
year. In an attempt to assist the states with the development and imple- 
mentation of local Career education programs, in fiscal 1972 and fiscal 
1973 the Commissioner turned over to the states $9 million of the Com- 
missionefs share foe the purpose of establishing a career education site 
in each state. 

Part D i 

The 1968 Amendments authorize Part D funds to support exemplary, 
pilot, and demonstration projects at elementary and secondary school 
levels. Part D funds.are also divided equally between the Commissioner 
and the states. A major eniphasis of the Commissioner s share of Part D, 
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which is 50 percent of the total, has been career education. The principal 
objectives of Part D projects have been to familiarize students with occu- 
pations and to provide work experience programs, guidance, and coun- 
seling. Part D projects have served as demonstration sites within each 
state, providing practical, operating examples of career education. 

As required by law, the federally administered Part D projects are 
geographically distributed in suoh a way that each state has at least one 
project. Additionally, a populr^ion formula ensures that larger states 
receive larger awards. Most projects are funded for a three-year period. 

A state's share of Part D funds is usually granted by the state to local 
school districts or individuaHpcal schools for exemplary programs. In 
some: stales, the state Rart D fui^(^^fe combined with those of the feder- 
al share to support one project. / ' 

An the first found (fiscal 197&ibrough fiscal 1973) of Part D funding 
from the Commissioner's share, at least one project in each state was 
funded for a three-year period, although not all projects began during 
the first year. The first round of funding supported 66 projects in all, and 
the second round (fiscal 1973 through fiscal 1976) supported 52 projects. 
Allocations were approximately $100,000 to $200,000 per state per year, 
b'it obligations were usually considerably less than allocations. 

Between fiscal 1970 and fiscal 1972, the states' share of Part D sup- 
ported between 300 and 410 projects per year at a median cost of ap- 
proximately $130,000 per project. (Development Associates calculated 
median rather than mean funding level due to the skewed distribution of 
funding levels.) Thus, both state and Commissioner's Part D projects 
A .ir; funded at approximately the same level. 



Pari I 

Fart I funds are used to promote the development and dissemination of 
wcativ >nal education curriculum materials, to develop standards for cur- 
riculum development, to coordinate state efforts and prepare current lists 
of available materials, to survey curriculum materials produced by other 
govfeirnmem agencies, to evaluate materials and their uses, and to train 
personnel in curriculum development. 

Nearly all of the Part I money has been spent on curriculum develop- 
i.icnv and dissemination. Very little has been spent on the establishment 
cf standards for curriculum development or on evaluation of the materi- 
als developed. 

Unlike Parts C and D, all Part J funds, are federally administered. 
Most activities are carried out through individual projects developed in 
response to requests for proposals. The U.S: Commissioner of Education 
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is authorized to make grants and contracts with colleges and universities, 
state boards of education, and other public and nonprofit private agen-v 
cies and institutions, as well as profit-making institutions^ Project dura- 
tion ranges from one to four years. Because there are no geographic 
restrictions on Part. I awards, there has not been one Part I project -in 
, each state each year," Between, fiscal 1971 and fiscal 1974, Part I funds 
supported between 13 and 28 projects per year; the average award size 
ranged from $122,000 to $391,000. 

In 1972 the Office of Education developed 15 occupational clusters to 
guide curriculum development: agribusiness and natural resources, busi- 
ness and office, communications and media, construction, consumer and 
homemaking, environment, fine art? and humanities, health, manufaCr. 
turing, marine science, marketing and distribution, personal services, 
public service, recreation and hospitality, and transportation. This repre- 
sented a significant shift from the more traditional, notion of seven pri- 
mary vocational education areas: agriculture, distributive education, 
health occupations, home economics, office occupations, technical edu- 
cation, and trade and industrial occupations. Between fiscal 1972 and 
fiscal 1974, more than half of the Part I currictilum development projects 
addressed two or more of the 1 5 occupational clusters. 

CONCLUSIONS 

Several broad generalizations can be drawn from the information pre- 
sented in this chapter. First, there has been- more development and dem- 
onstration than research in the past decade of vocational education r&d. 
In addition, most early research was largely descriptive rathtr than ex- 
perimental.* In the mid-1960s there were few vocational euucation per- 
sonnel who knew how to conduct research projects and *tl|ie social sci- 
entists involved knew little about vocational education. 
■ Second, career education has been heavily emphasized, especially in 
projects supported by funds from Parts C and D. In 197hr4t the Com- 
missioner's direction, usoe invested 57.5 million of Part C and D funds 
in the development of six large-scale, school-based, career education 
models. Alternative approaches to career education were also funded in 
fiscal 1971; employer-based, home/community-based, and residential- 
based career education models were heavily supported. (In fiscal 1975, 
for the first ^ime, a separate career education budget of nearly $10 mil- 
lion was approved by Congress.) . 

Third, there has been much activity in curriculum development, al- 
though Part I has always been the smallest of the three parts. Part C has 
supported the development and dissemination of many curriculum mate- 

^ ' . \ 
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rials. Part D has also supported demonstration sites to aid in the installa- 
tion of new curricula. 

It should be noteq that many of the data that would have helped this 
Committee describe the vocational education R&D program were not 
available. For example, only project titles, name of grantee or contractor, 
and amount of award were available for state Part C.projects; even short 
abstracts of the projects could not be obtained. Final reports^ when sub- 
mitted, are sent to eric, where they may or may not be reproduced, and 
stored by USOE in permanent, relatively inaccessible files. If this informa- 
tion had been readily available, the Committee could have prepared a 
more thorough description and assessment of the r&d programs. 
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What has been gained from the $250 million invested by the U.S. Office 
of Education in vocational education ,r&d over the past decade? The 
Committee's assessment of the vocational education r&d program fo- 
cused on two questions: What contributions have been made by r&d to 
knowledge about vocational education? What has been the impact of 
R&D on students? Both of these questions have been difficult to answer. 

In its attempt tq identify the accumulation of knowledge on various 
topics related to vocational education, the Committee was hampered by 
the absence of reports on proje . ts funded by usOE. Because the work 
funded only by usOE could not be reviewed as a separate body of litera- 
ture, the Committee reviewed research related to vocational education- 
regardless of funding agency. (The results of that review are summarized 
in Appendix A, and a related discussion of research issues is presented in 
Chapter '2.) In looking for evidence, of impact of r&d on students, the 
Committee and staff interviewed many people involved in vocational 
education and its R&D, conducted hearings, and reviewed evaluation re- 
ports. (The individuals and organizations that were contacted are listed 
in Appendix B.) * 

R&D conducted by federal agencies to address social problems can 
serve primarily either the agency or the practitioners throughout the 
country. For example, the Department of Labor conducts its manpower 
R&D program to produce information for its own use and to help solve 
national manpower problems, while the vocational education r&d pro- 
gram in the Office Of Education serves primarily the needs of practition- 
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ers. USOE allocates r&d funds by state in order to meet the differing needs 
of the numerous vocational education programs, which are operated by 
state and local education agencies. Evaluation of vocational education 
R&D is quite difficult because of the complexities of the educational and 
R&D programs: the thousands of r&d products should be evaluated ac- 
cording to the differing needs of practitioners and in settings appropriate . 
**■ ' 'ftrMl^dr intended use. 

AssessiTtg^he impact of the R&D-program on students has been difficult 
because thereare virtually no data that permit a rigorous e^valuation of 
all the R&D and its outcomes. Project evaluations have been rare, and the 
impact measures used have be^aJ5?eak, superficial, and Iransilory. Eval.u- 
ation of demonstration projects has been especially problematic, partly 
due to lack of criteria for measuring success. In addition, follow-up or 
longitudinal data on graduates from various vocational education and 
other programs have not been collected. The Committee has therefore 
based its assessment of vocational education R&D»on incomplete and 
sometimes subjective information, program evaluations, and project de- 
scriptions. 

The Committee had difficulty finding evidence that r&d has had a, 
measurable impact. Although we. do, know that some r&d products have 
been disseminated and used by practitioners, we do not know what pro- 
portion of the total they are or what proportion is either reasonable or 
desirable.' Further, we do not know how well these products fit the needs 
of] potential or actual users. Even complete information about implemen- 
tation of R&D products does not accurately reflect impact. Since research 
often involves testing ideas or products, discovering that a promising 
idea or product is, in fact, unworkable is still a valuable contribution. 
Many ideas or products should not be implemented or adopted, and 
researchers perform a valuable service by finding evidence that'prevents 
/wasteful application. 

j In short, there are no hard data to substantiate the belief that the 
Ivocational education r&d prompted by the legislation of 1963 and 1968 
I has improved vocational education. Even though there are other benefits 
I from R&D (such as accumulation of knowledge, building r&d capability) 
these have rarely been measured, documented, or validated. 

I LARGE-SCALE EVALUATIONS OF R&D 

J There have been major efforts by two organizations (Project Baseline 
/ and Development Associates, Inc.) to evaluate broad impact of vocation- 
al education r&d. Project'Baseline, located at Northern Arizona Univer- 
sity, is funded under the federal share of Part C of the 1968 Amendments 
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to collect data and report on several aspects of vocational education in 
the United States. Project Baseline, which usually receives over $200,000 
yearly, has produced four comprehensive annual reports and several sup- 
plemental reports on such topics as preparing vocational education 
teachers, women in vocational education, and the impact of manpower 
training programs on the labor market. Project. Baseline attempted to 
evaluate state-administered Part C and D projects and federally Adminis- 
tered Part D projects; this evaluation of r&d is just one of their recent 
activities. Development Associates, Inc., a private managenient consult- 
ing firm specializing in evaluating social programs, attempted-to evaluate 
Part D programs, focusing more sharply on the federal than the state 
projects. These evaluations have had only limited success and produced 
rather narrow and superficial definitions of impact. There is not more 
evidence of impact partly because impact is not usually immediately 
manifest, and this Committee is studying a time spah that is both short 
and recent. * , . 

The Committee presents the findings of Project Baseline and Develop-* 
ment Associates both for their inherent value and to explain the need for 
more and higher-quality evaluative data. These two projects present the 
only large-scale evidence of impact of vocational education r&d known^ 
to the Committee. However, the reader should be cautioned that the data 
collected by these organizations and presented here are not adequate for 
a rigorous evaluation. Project Baseline surveyed people who had been 
deeply invoK:d in the projects evaluated, so the probability of judg- 
ments biased in favor of the r&d is extremely high. Development Associ- . 
ates' conclusions are tenuous partly because of problems in obtaining 
unambiguous enrollment data on the students sampled. 

PROJECT BASELINE ' ^ ' 

Project Baseline attempted to determine the impact of research, develop- 
ment, and exemplary projects on vocational education funded under the 
sta^-administered Part C program, the state-administered Part D pro- 
gram, and the first round of the Co^nriissioner's share of the Part D 
program. In Project Baseline's Fourth National, Report (Lee 1975)-the 
term "impact" is used to mean the extent to which local districts -have 
maintained and implemented previously developed products and results. 
This measure of impact was used to assess project outcomes because 
Baseline did not have nor could it obtain accurate measures of changes 
in student skills 'and pther evaluative data. 

The impact of federally administered Part D projects is classified in 
five categories: \^ - / 
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1. Implementation in many states 

2; Extensive implementation within a state (Extensive is operationally 
defined as enough acceptance that the majority of schools,in..the state are 
using some or all of the concepts, materials, and other products.) 

3. Some implenrientaticm in several areas of a state 

4. Implementation within a district 

5. Minimal or no implementation 

A similar classification scheme is used for state-administered projects': 

^ . . . . , . 

1 . Wide implementation or extensive use within a state 

2. Implementation in the original site .^nd in a few additional sites 

3. Implementation'only in the original site 

. 4. Use of results as a basis for additional work 
5. Little or no known current use 

An effort was made by BaseUne to assess 54 federally administered 
Part D projects by reading final reports and interviewing project di- 
rectors by telephone. Interview questions solicited information regarding 
the extent of the dissemination and utilization of R&D results and prod- 
ucts. Baseline found that 40 of the 54 projects fell into the first four 
categories;- they "have had some impact in bringing about changes in at 
least a few additional school districts within their states and in additional 
schools within their own districts" (Lee 1975, p. 61). Twelve of the 40 
projects have had extensive impact within the stati? (category 2), and four 
have been adopted in whole or part by other states (category 1). 

Stratified random samples of state-administered Part C and D projects 
were selected for assessment by Project Baseline. Stratification of Part C 
projects was based on statewide versus local focus and whether or not 
impact was primarily on policy, administrative practices (including guid- 
ance), or instruction. Part D projects were stratified according to the 
following topics: "cooperation between education and manpower, post- 
secondary education programs directed toward out-of-school youth, and 
programs for young people in school'' (p. 63). Information about projects 
was obtained *by telephone contact with research coordinating, unit di- 
rectors, project directors, or school district personnel. Respondents were 
asked about the extent to which "projects, methods, and results had been 
implemented in additional schools or school districts. There was no fol- 
^ low-up to determine the duration of newly adopted programs or prod- 
ucts or to verify the responses independently. Baseline found that 94 
percent. of the sample of 96 state-administered Part C projects had had 
some impact, at least locally' (categories 1-4); 29 percent were "widely 
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implemented or extensively used" (category . 1). There were similar 
findings for state-administered Part D projects: 97 percent had some 
impact (categories 1-4) and 20 percent were widely implemented or used 
(category 1). . 

Problems in obtaining student outcome information wer^ experienced 
in the Project Baseline evaluation. The lack of baseline data made it 
impossible to assess accurately indicators of student growth and prog- 
ress. Distead, the study describes several state-administered Part C and D 
projects that were^judged by state r&d personnel to have had the greatest 
impact. Again, success^ppears to have been determined by the extent to 
which the product is used by other schools, generates interest among 
students, teachers, and the community, and provides information for use 
by teachers ^nd administrators. The benefits of using research results 
were not assessed. 



DEVELOPMENT ASSOCIATES 

In their evaluation. Development Associates attempted to assess the Part 
D program for USOE, using pre- and post-treatment measures of career 
development. For each of the 50 projects selected for evaluation, experi- 
mental groups were drawn from student§,'in grades 6, 9, and 12 partici- 
pating in Part D programs, and control groups were drawn from students 
not participating in those programs. The study was hampered by difficul- 
ties in sample identification: "... most projects were not ^le to iden- 
tify precisely either the students who were 'participants' in the program 
or what constituted 'participation'" (Development Associates 1975, p. 
23). 

Questionnaires and tests of student outcomes were given to experi- 
mental and control groups. The 13 questions used to measure student 
outcomes included: 

Are student participants able to identify a greater number of occupa- 
tions than non-participants? 

Do students demonstrate more familiarity with tasks and functions 
associated with selected occupations than the comparison groups? 

Are a greater number of students who have graduated from school and 
who participated in the Part D project employed full-time or engaged in 
further training than students who did not participate? 

The questionnaires produced mixed results. Five questions were elimi- 
nated from the summary due to factors such as insufficient or invalid 
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data. Significant differences between participants in the Part D projects 
and non-participants were found for only soine grade levels on only 
three outcome questions. In light of the problems of identifying the study 
sample as well as the inconclusive findings, it is impossible to conclude 
that the Part D projects significantly improved student outcomes. 

Several of the problems of evaluating vocational education r&d after 
projects have been completed are evident in the.work of Development 
Associates: participation in the. projects was hard to define since it is not 
a simple yes/no variable. Because of inadequate records, administrators 
had difficulty in determining which students had participated in the pro- 
jects; the measures of impact that could be collected were somewhat 
weak. In addition, placement and follow-up data, which Development 
Associates did not have, can be collected only with expenditures of much 
time and money. Even though the study appeared to be very carefully 
planned, it could not overcome the shortcomings imposed by these other 
factors. 

EXAMPLES OF PROJECTS JUDGED TO BE SUCCESSFUL 

Definitive conclusions on the whole body of vocational education r&d 
cannot be drawji on the basis of the two national evaluations described 
above. Therefore, in an attempt to find some substantiation of the impact 
of R&D, the Committee analyzed- individual projects on the basis of their 
descriptions. 

The Committee was able to locate a number of projects with some 
measure of impact. Six projects judged to be successful were drawn from 
lists compiled by usoe and the Southwide Research Coordinating Coun- 
cil; other examples were supplied by Committee members and a consul- 
tant. The nine examples arc described here. 

U.S. OFFICE OF EDUCATION 

Eighteen federally funded projects were cited by Howard F. Hjelm and 
Glenn C. Boerrigter (1974) as examples of visible and useful accomplish- 
ments of vocational education r&d over the past decade. Three projects 
taken-from their list are described below. The Commit'tee selected these 
projects as examples because they demonstrate three different objective 
measures of impact. 

In the initial phase of the Aviation Mechanics Project (reported in 
Allen 1968). a core curriculum for aviation mechanics was identified. 
During Phase IK airlines and aviation companies were surveyed in order 
to identify industry requirements. Phase III consisted of two parts: Dur- 
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ing Pan I, 100 teachers were involved in curriculum development and 
teacher training. During Part II, 30 percent of the companies surveyed in 
Phase II were resurveyed. The second survey showed the necessity for 
readjusting school curricula in major tasks such as painting and welding. 
As a result, the core curriculum was updated and a method by which the 
Federal Aviation Administration could continue periodic updates was 
established. Recommendations made on the basis oi the second survey 
reflect current requiremeiits of the aviation industry. Establishing a 
method for industry to update vocational curricula may provide more 
relevant training for vocational students, but the actual improvement in 
student job prospects and the advantage to employers in hiring graduates 
trained with this curriculum have not been measured. 

The objective of the Electro-Mechanical Equipment Technology Proj- 
ect (reported in Roney 1971) was to assist tworyear colleges in establish- 
ing training programs by developing the necessary planning and instruc- 
tional materials and by providing direct program planning assistance. 
Researchers developed and tested an integrated system^ of instruction 
built around discrete technical concepts that are basic to more than one 
technology. Student achievement was measured and recorded at all 
stages of the project. Instructional materials were tested on students and 
revised. Case studies were written for the purpose of identifying adminis- 
trative problems. By the end of the project, materials were being used in 
30 states and planning assistance had been provided to 375 schools. Ma- 
terials were disseminated widely after the project was completed. 

Evaluation oHthe electro-mechanical equipment curricula showed that 
students successfully learned .skills and knowledge in electro-mechanical 
equipment technology. However, it is not clear how many of the students 
who used the curriculum materials were placed in jobs or how many 
used the acquired skills and knowledge on the job. The success of the 
project cannot be fully determined until it can be shown that the stu- 
dents trained in electro-mechanical equipment technology gained an ad- 
vantage in finding and keeping good jobs because of their training. 

"The Kingdom of Could be You'' (reported in Sutherland Associates 
1974) consisted of 16 short cartoon and real-life films related to occupa- 
tional clusters. The project had three objectives: to develop awareness of 
future job opportunities in young children; to enlarge the vocational self- 
concept by encouraging children to see themselves in a variety of occu- 
pational roles; and to engender a work ethic in children. The films were 
originally presented on the Captain Kangaroo Show. 

The film evaluation project involved 124 children, three to seven years 
of age, with appropriate ethnic, sex, and socioeconomic'iJistribution. Pre- 
and post-tests were used to measure gain in career awareness. Children 
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showed a remarkable gain in awareness of occupational opportunity as 
judged by comparison of interviews before and after exposure to the 
films. The 16 films are available from a commercial distributor. 

SOUTHV^lDE RESEARCH COORDINATING COUNCIL 

The Southwide Research Coordinating Council, consisting of the re- 
search coordinating units in 14 states in the southeastern United States, 
compiled a casebook of r&d activities they judged to have had impact 
(1975). The report is not intended to assess all vocational education R&D, 
but to demonstrate that some projects have had a positive impact in the 
Southeast. Although the sources of information are not explicitly de- 
scribed in the report, it appears that project directors or staff members of 
projects supplied impact data on their own projects. Therefore, the ob-^ 
jectivity of the information is open to question. A sample of three of 
these 26 projects demonstrates the kinds of projects included and ways in 
which impact is defined. 

The Alabama Vocational Management-Information System was de- 
signed to provide manpower data for management of vocational educa- 
tion programs in the state. In addition, an accounting system provides 
fiscal data as well as student enrollment, completion, and follow-up in- 
formation by teacher, school, program, and occupational objective in 
order to evaluate programs. As a result, an increase in planning activities 
has been noted as the data are made available to ass'st local planning 
efforts. Special data requests from teachers, administrators, and planners 
have indicated that the data are needed, and the. management informa- 
tion system is reported to have resulted in more accurate and useful 
information. However, the accuracy of the data supplied by the system 
has not been evaluated objectively. 

A Part D project in Florida, "An Exemplary Model for a Total Eco- 
logical Approach to Non-graded Vocational Programs in Separate Edu- 
cational-Centers," was designed to improve students* attitudes toward 
their environment, iniprove academic achievement, and develop occupa- 
tional skills, as well as to develop new techniques for teaching disadvan- 
taged students. Project staff reported that achievement measures indicat- 
ed that all of these objectives were met. This project was the forerunner 
of career education in the state, and eventually the model was adopted 
by all education districts in the state. 

A career education project in Mississippi exposed students to occupa- 
tional education in elementary school and continued preparing students 
for the world of work through high school and post-secondary school. 
An evaluation using experimental and control student groups showed 
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that students in the project had more positive attitudes toward work and 
a greater knowledge of careers than did students in the control group. 
Project staff reported that the project's career education methods have 
spread to 22 school systems in the state. 

Mostjof the other projects listed in the Southwide Research Coordinat- 
ing Council's report have impact measures similar to those in these three 
examples. For most placement services, management information sys- 
tems, and evaluation models, adoption by a certain number of schools or 
school districts is reported as the measure of impact. Impact of curricu- 
lum materials and instructional techniques is reported by adoption and 
by the number of teachers and students involved. Occasionally, the re- 
port of a third-party evaluation is cited; The measures of impact that are 
reported are generally not objective and do not convey information 
about changes directly related to students. 

OTHER EXAMPLES 

Some projects can be considered successful if research findings are used 
to change program policy. An example was provided by a member of 
this Committee, the Texas Director of Vocational Education. A Part C 
project- entitled "A Survey of the Occupational and Educational Needs 
of the American Indian in Dallas County" was designed to collect in- 
formation for use in program planning. Findings indicated specific edu- 
cational needs, and on the basis of recommendations in the final report, 
the Dallas County Community College District (1973) initiated a cooper- 
ative education program for Dallas County Indians. Study findings also 
provided a basis for the development of a pre-school program and a 
youth program for Indians later funded by usoe. The Inter-Tribal Center 
and Clinic also used the findings to develop a manpower training pro- 
gram funded by the Department of Housing and Urban Development. 
Finally, as an outgrowth of the study, the Dallas County Community 
College District began operating three outreach programs designed to 
recruit and counsel American Indian students and other minority group 
members. However, the impact of the pre-schoo), youth, manpower 
training, and outreach programs has not been studied in formal evalua- 
tions. 

A nationwide project on allied health occupations (reported in Ander- 
son 1973 and Fielstra 1973) developed, validated, tested, and disseminat- 
ed instructional material for more than 20 allied health occupations. 
Nationwide conferences and national advisory committees were in- 
volved in developing the mstructional material. Twelve of the final in- 
structional material documents were published by commercial publish- 
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ers. One of the publications (nursing) has sold over SO^OOO copies, and 
another (dental assisting) has sold nearly 35,000 copies. These materials 
have been used by every state in the nation and by 35 foreign countries. 
Other materials not published commercially are in great demand three 
years following the completion of the basic project. Again, however, stu- 
dents trained with these materials have not been followed up to deter- 
mine how helpful the materials are in preparing students for jobs. 

Krumbollz et al. (1967, 1968) constructed "job experience kits" in 
accounting and six other occupations in order to test the social learning 
theory that career interests are learned as a result of successfully master- 
ing occupationally related tasks. The kits were designed to stimulate 
young people to explore career opportunities by exposing them to simu- 
lated occupational tasks and easily solved problems. In one of the studies 
the theory was tested by administering the accounting kit to a random 
sample of high school students and giving control groups other informa- 
tion about accounting or general occupational information. Follow-up 
interviews and questionnaire responses revealed that interest in account- 
ing was increased by the problem-solving accountant kit significantly 
more than by other treatments. Interest in 41 other occupations changed 
no more than would be expected by random fluctuation: however, the 
experience of using the accounting kit seemed to stimulate later inquiries 
about other occupations. Other experimental results showed that prob- 
lem solving was an effective method of stimulating interest, particularly 
for students from communities of low socioeconomic status. The devel- 
opers of the job experience kits contracted with an outside agency to 
develop kits in 13 more occupational areas and. to distribute them na- 
tionally. The kits have been distributed in every state. 

CONCLUSIONS 

Although the examples of projects cited here represent only a fraction of 
the thousands of projects funded, they demonstrate some of the potential 
contributions of vocational education R&D to vocational education. The 
actual contributions have not been well-documented: most of the proj- 
ects described as successful have not been rigorously evaluated. Often, 
they are said to have impact because research products are ''widely disse- 
minated'' or "interest is generated'' among students and teachers. Proj- 
ects arc also said to be successful if their reports are published commer- 
cially and large numbers of copies are sold. It should be noted that few 
of the projects cited as successful have addressed the needs of students in 
secondary schools, where vocational education enrollments have been 
concentrated. 
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Rarely has it been shown that students have benefited from the proj- 
ects cited as successful. In most cases it cannot be determined that r&d 
results or products are actually used, and, if they are used, it cannot be 
determined whether they have any significant or long-lasting effects on 
students. The lack of evidence implies that much vocational education 
R&D has probably not had a widespread effect on classroom activities 
and student outcomes. 

PERCEPTIONS OF R&D EFFECTIVENESS 

In the interviews and hearings conducted by the Committee and staff, 
respondents were requested to give information on the impact of r&d on 
various groups of vocational education consumers, especially students. 

Virtually all of the people interviewed had great difficulty in judging 
the effectiveness of the research and, development familiar to them. The 
effects of R&D are diffuse: r&d may affect the work and attitudes of 
federal and state administrators of both vocational education and r&d, 
other researchers, teachers, other school personnel, students, and em- 
ployers of vocational graduates. All respondents felt it nearly impossible 
to trace changes in students' acquired abilities and attitudes to research 
projects. Most could cite advances in vocational education that had ac- 
companied progress in research but were not necessarily caused by re- 
search. Some improvements that were mentioned often are an increase in 
student enrollment, "better" and "more meaningful" programs, im- 
proved teacher attitudes, and greater student enthusiasm. 

One research coordinating unit director stated that vocational educa- 
tion R&D has enhanced the stature of other types of research in his state; 
people recognize that research can be practical and is, therefore, valu- 
able. There have been similar improvements in the image of vocational 
education as a result of increased awareness of the field by the public. 

Several people have noted that research may have benefits that are less 
visible than the accumulation of knowledge about particular topics. 
Scientific research is often described as a way of testing existing hy- 
potheses, but it can also raise new questions, or help to map unexplored 
territory as well as test existing hypotheses. Researchers can reformulate 
old questions on the basis of new evidence. ^ 

Some of the people interviewed have observed innovative demonstra- 
tions and development programs and believe them to be more interesting 
than traditional programs. In one state, the teaching efficiency in a nurs- 
ing program has increased due to an innovative project, resulting in a 
shorter program covering the same amount of material. This sort of 
measure is much more quantifiable than many, for example, measures of 
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suidenls' enthusiasm or the effectiveness of programs. Some of those who 
often visit the schools are convinced that exciting things are happening 
partly as a result of vocational education R&D efforts. It is as difficult to 
refute such statements as it is to use them as hard evidence that r&d has 
made a difference. 

Virtually no hard data on the impact of r&d projects were presented at 
the hearings conducted by the Committee. Some witnesses noted that 
impact is very difficult to assess since many factors, not just R&D, affect 
events. Some noted that knowledge has increased in certain areas, but 
they were not asked to substantiate such statements and did not do so. 
One local administrator stated that he had observed very little impact as 
a direct result of R&D. He noted that many local vocational education 
administrators share his perception of R&D. Most of the witnesses, how- 
ever, were very enthusiastic about R&D and its value in improving voca- 
tional education. 

The mere existence of vocational education R&D is felt by many to 
have had some notable benefits not specifically related to changes in the 
structure and content of vocational education and the r&d program. 
Funding for vocational education R&D has most certainly helped develop 
research capability and increased the pr<Dminence of vocational educa- 
tion. Research personnel have been drawn to vocational education, vo- 
cational educators have gained experience in research, and a community 
of vocational education researchers has developed. In addition, research 
institutions supported by r&d funding have added to the visibility of 
vocational education R&D and have provided a setting thai potentially 
increases research capability. Most of those interviewed noted changes in 
the supply of useful data as a result of research. Perhaps equally impor- 
tant is the seldom-noted fact that as the supply of information increases," 
administrators tend to rely lesson intuition and more on data. Finally, as 
the data supply and demand increase, so do the standards forjudging the 
" quality of research. 

LACK OF EVALUATIVE MATERIAL 

■ There are no evaluative data or even judgmental statements that would 
make possible a comprehensive assessment of the vast majority of voca- 
tional education R&d over the last ten years. A comprehensive evaluation 
is virtually impossible for two major reasons. First, the priorities and 
objectives of vocational education R&d have been poorly defined and 
have rapidly changed so that there are no goals against which to measure 
actual accomplishments. Second, there has not been a consistent, broad- 
ly based, and long-term concern with collecting, analyzing, and reporting 



40 ASSESSING VOCATIONAL EDUCATION RESEARCH AND DEVELOPMENT 

evaluative data on r&d products and their effects on students and society 
in general. 

Rigorous evaluations and follow-up studies are not only costly, but 
quite difficult to conduct. As is the case with all social science research 
imposing laboratory conditions on real-life situations is all but.impossi- 
• ble for both ethical and economic reasons. Project evaluations in voca- 
tional education typically have been self-reports or post hoc, third-party 
reports rather than evaluations built into project designs. There have 
been very few follow-up studies to determine the eventual impact of r&d 
projects on students. The problems associated with evaluation are by no 
means unique to vocational education, Egon G. Guba explained many 
shortcomings of the methods of evaluation, but expressed optimism that 
\c problems would soon be solved (Guba 1969, p. 29): 

^ The American educational establishment is currently makigg a massive effort at 
self-improvement. Unprecedented resources, steniming mainly from the federal 
govermnenl . . . , are being expended on a variety of promising but as yet un- 
proved programs. To assure the effective and efficient uses of these resources, 
and, even more importantly, to determine the real utility of the innovative ap- 
'proaches, it is necessar}' to. gather hard data about their performance. Evalua- 
tion is the process best suited for this purpose. 

The traditional methods of evaluation have failed educators in their attempts 
to assess the impact of innovations in operating systems. Indeed, for decades the 
evidence produced by the application of conventional evaluation procedures has 
contradicted the experiential evidence of the practitioner. 

Advances have been made in evaluation technology since Guba's arti- 
cle was published in 1969, but there are still problems to be solved, and 
the advances already made must be put into practice. More recently, 
Jerome Moss, Jr., and Erast W. Stromsdorfer (1971, p. 261) concluded: 

It is apparent from this review of studies, both non-economic and economic,, 
that the methodological issues facing any analysis of the effects of vocational 
education are formidable . . . [I]t is our opinion that the existing analyses, tak- 
en as a whole, do not use effectively the methodological knowledge which is cur- 
rently available. For instance, the concept of use of i control group would ap- 
pear obvious; yet in studies which pose hypotheses whose testing clearly implies 
the need for a control group, none is employed. And, in other evaluations, 
where recommendations are made which involved extensive commitment of eco- 
nomic resources, there is no treatment of costs along with benefits. These are 
only examples. Unfortunately, they do not represent uncommon errors. 
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CONCLUSIONS AND RECOMMENDATIONS 

i 

CONCLUSIONS 

The Committee's assessment of vocational education r&d has been limit- 
ed because a collection of final reports of all R&d projects is not avail- 
able. Further, data for program evaluation have not been collected. 
Therefore, the Committee's assessment has been based on partial evalua- 
tions conducted -by others, small-scale surveys of researchers and admin- 
istrators, some reports of r&d projects, impressions of acknowledged 
experts, andjudgments of Committee members. The Committee has con- 
cluded that vocational education r&d of the past decade has not- had. 
documented, widespread impact. The available data do not indicate tljat 
vocational education r&d findings and products have had an influence 
on the knowledge, skills, or employability of large numbers of students. 

There are insufficient data to allow for a comprehensive evaluation of 
vocational education or its supporting r&d. Erogram success has most 
"onen been measured in terms of initial job placement, andiittle attention 
has been given to assessing the effects of programs over an extended 
period of time. The- impact of r&d has been 'measured most often in 
terms of user acceptance, defined as frequency of recjuests for informa- 
tion rather than frequency of implementation of findings and installation 
of products. Impact measures have often been subjective and difllicult to 
validate. They have sometimes been superficial and have failed to cpn- 
^sider the long-term consequences of vocational .education prqgrams or 
R&p. In addition, the quantified effects of different programs or projects 
oft^n cannot be compared because measurements were not standardized 
across projects. 

The Committee recognizes that vocational education r&d is relatively 
new and\that it faces many of the difficulties of all educational research, 
and, in fact, of all social science research. One difficulty is that models 
for evaluation are not readily available. Further, the effects of r&d proj- 
ects cannot be isolated -..many social, psychological, and economic fac- 
tors can confound or weaken the impact of r&d. 

Despite these difficulties, the Committee believes that vocational edu- , 
cation r&d has adcied to the body of knowledge about vocational educa- 
tion and its students^ Vocational education r&d has also produced new 
programs and classroom techniques for use across the nation. Many 
curriculum materials have, been published commercially and have been 
purchased by large numbers of people, although the extent to which 
these products or other research results are beneficial or are actually used 
by practitioners cannot be adequately determined at this time. Funding 
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for vocational education r&d has also increased research capability. Na- 
tional and state institutions exist to facilitate and coordinate the conduct 
of R&D, and numerous researchers have been trained in or drawn to 
vocational education. 



RECOMMENDATIONS 

The Committee recommends severa- interreb.ted strategies that are nec- 
essary for a comprehensive evaln,ation of vocational education rad. 
First, the goals 'and objectives cne r&d program, which should logical- 
ly follow from the objective? of vocational education, should be clearly 
defined. Only then can evatuation measure the degree of success in. at- 
taining these objective: i. tbe U.S. Commissioner of Education and state 
directors of vocatk aai education should begin to fund studies of the 
objectives and pK jrinos of vocational education programs and related 
work. 

Second, in order to identify in the future contributions made by Rs&d, 
funding agencies should be able to provide access to final reports of all 
R&D projects. Research synthesis documents would be especially helpful 
in reporting and assessing what has been iearhed. 

Third, USOE should develop, with ample input from researchers and 
practitioners, a plan for evaluation of r&d that includes the collection of 
longitudinal data. A plannjed mix of self-evaluation, agency evaluation, 
and third-pafty evaluation is suggested. In general, post hoc evaluations 
should be avoided. A sample of r&d projects should receive funding for 
evaluations, planned at the same tj-me as the projects themselves. Not all 
r&d projects need extensive external evaluation: a few have adequate 
evaluation built into the research design, and others are so small that 
adequate evaluation would cost more than the original research. 

USOE should determine exactly what factors are to be measured by 
evaluations. Presently, impact on the knowledge, skills, and employabili- 
ty of students is the most widely accepted factor. Also of interest are the 
;. r:?lifications of the researchers; factors related to research design (quali- 
ty and appropriateness, , implementation, data analyses, interpretations, 
■ and the extent of validation of results); the methods of dissemination; 
and the degree of utilization of products. A comparative, evaluative data 
base is needed for determining what vocational education programs and 
R&D have been effective and should receive continued -support. Evalua- 
tion criteria should be studied^and standards developed so that evalua- 
tions of different projects would be as nearly comparable as possible, 
given their inherent differences. However, important questions that are 
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unique to certain programs or projects should not be eliminated simply 
. because of their uniqueness. * , . 

In order to measure the effectiveness of vocational education pro- 
grams, USOE should collect long-term, follow-up data on a national sam- 
- pie of 4II people who enroll in vocational education programs and of 
comparable people who do not enroll. The data collected should include 
measures such as job satisfaction, upward job mobility, wages, satisfac- 
tion of the employer with the worker, and continuation of education 
(vocational as well as academic, both degree and non-degree). If longitu- 
dinal data were collected on new and traditional vocational programs, 
their differential effectiveness could be studied. Finally, long-term, longi- 
tudinal studies of potential client groups could be a useful, albeit expen- 
sive, means of gathering data and might be used by usoE for evaluation 
of vocational education. / 

Because vocational education r&d is relatively new and rapidly chang-^ 
ing, the ongoing program of R&D should be supplemented by the work of 
an advisory panel charged with studying all of vocational education, 
including its R&D. This panel should be convened every five years to 
assess the acqomplishments and failures of vocational education and to 
make recommendations in terms of goals and objectives for the future. 
The broad, long-range perspective of such an advisory panel would 
strengthen and give guidance to continuing work in vocational education 

R&D. 

. \ . 

k - ■ ' 
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Vocational \ 
Education 
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Vocational education R&D has not had as great an impact as it could 
have had partly because of certain characteristics of the administration 
■of the R&D program. In this chapter, the Committee reccEmmends several 
changes in the administration of the program that are intended to im- 
prove the resulting r&d. Before these recommendations are presented, 
however, the structure and management of the.'R&D program are de- 
scribed. ' ■ . . 

The administration of the vocational education r&d program is com- ' 
plicated by the need to accommodate two major factors: three categories 
of R&D (research and Jevelopment, demonstration, and curriculum de- 
velopment), and three levels of organization (federal, regional, and state). 
This chapter discusses, the administration of .the vocational education 
RAD program with respect to planning; administration^ and management, 
outlining the roles of the major national, regional, and state organiza- . 
tions: Jhe U.S. 'Office of Education, the two national r&d centers, the 
National Advisory Council on Vocational Education, USOE regional 
offices, the National Network for Curriculum Coordination in Vocation- 
al and Technical Education, state' education agencies, state research co- 
ordinating units (Rcusj, and State Advisory Councils. (See Appendix B 
for a discussion of sources of information.) 

After the organizations involved in the administration of the r&d pro- 
gram are described, a separate discussion^of disseniination and utilize- . 
tion of R&D products is presented. The. problem-^ of dissemination and 
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utilization involves many difTerent^tyP^s of organizations and. so is treat- 
ed separately. ^^y^^"^""^ 

NATIONAL INSTITUTIONS 

U.S. OFFICE OF V )UCATlON 

The U.S. Office of Education of the Department of Health, Education, 
and Welfare plays the major role in the administration of the vocational 
education r&d program. It is responsible for overall planning, including 
coordination of Part I and of the federal shares of Parts C and D, and for 
setting priorities. It announces the availability of money for grants and 
contracts, reviews proposals received in answer to its requests for propos- 
als, and monitors projects once they are funded. usOE is also responsible 
for some dissemination of information and materials produced by those 
projects. It is becoming more concerned with evaluation of individual 
projects as well as evaluation of the entire vocational education r&d 
effort. 

' Although the Office of Education in Washington, D.C., has received 
some guidance from usOE personnel in the ten regional offices of HEw, 
the functions of the regional offices have not been well defined. Under 
President Nixon's move to decentralize the government, the regional 
offices were to be given increased authority, including some decision- 
making power for the administration of the Commissioner's share of Part 
D funds; however, after court rulings nullifying certain decentralization 
actions, decision-making authority was withdrawn from the regional 
offices. At the present time, the regional offices have very little responsi- 
bility for Parts C and I funds or projects: they can review applications 
for Part C awards and review requests for proposals for Part I awards 
prior to publication. 

The regional offices have a greater role in connection with Part D, 
participating in, planning and setting priorities. They review applications 
for Part D grants or contracts from states in their regions and send rec- 
ommendations to the federal office. Other regional office functions in 
connection with Part D include negotiating grants and awards, providing 
technical assistance and information to researchers, monitoring ongoing 
projects, participating in site visits for evaluation, closing out completed 
projects, and participating in federal office seminars and workshops. On 
the average, one person in each region is available for-.the.se Part D func- 
tions, and that person usually has additional responsibilities for other 
programs. 

\ 

\ 
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Administrative Location ^ 

Since 1964, the vocational education R&D program has been managed by 
many different divisions within the Office of Education. To administer 
Ihe research program, which came into existence as a result of the 1963 
Vocational Education Act, an Occupational Research and Planning Unit 
was established in 1964 within the usOE Division of Vocational and 
Technical Education. Branches within the Unit were established to man- 
age research in each of three substantive areas identified by usoe: em- 
ployment opportunities, human resources development, and education 
resources development and training. Table 2 lists changing administra- 
tive locations of the r&d program since 1964. At least partly as a result of 
these shifts in administrative structure, there is evidence of a rapid suc- 
cession of contradictory long-range plans. 



TABLE 2 Location of Federal Administration of Vocational Education 
R&D within the U.S. Office Of Education 



Fiscal Year 


Administrative Location 


1964 


Bureau of Adult, Vocational and Technical Education 




Division of Vocational and Technical Education 




Occupational Research and Planning Unit 


1965-1967 


Bureau of Research 




Division of Adult and Vocational Research 


1968* 


National Center for Educational Research and Development 




(bureau leve|> 




Division of Comprehensive and Vocational Research 


J969-I971t 


National Center for Educational Research and Development 




Di.vi.sion of Compreherr^^ :ind Vocational Research 




and 




Bureuu oi \dult. Vocation.Tl and Technical Education 




Div^ision of Vocational and TechnicaJ Education 


1972-1973 


Burerau of Adult» Vocational and Technical Education 




Division of Vocational and Technicali Education 


1974 


Buresiu of Occupiifional and Ai.'ijlt Education 




Divhion of Vocational Education R^esearch 


I975t 


Bureau u.t Occupational and Adult Edmcation 




Division of Research and Demonstration 



•This change is mostly a change M the tir'e of a group. 
fAdministration was split between ?)'e two divisions. 

^Thts change reflects only the renaming of u group. In this case, the major organizational 
structure and personnel involved remained the same. However, in other instances, espe* 
cially in fiscal 1964-196S and fiscal 1972-1974, there were major shifts in the personnel 
and structure involved in administration of the vocational education R&D program. 



56 



Administration of the Vocational Education R&D Program 



Coordination 

Since 1975 the Division of Research and Demonstration in the Bureau of 
Occupational and Adult Education (boae) has administered the federal 
half of the Part C research program, the federal half of the Part D dem- 
onstration program, and the Part I curriculum development program. 
The Division attempts to coordinate these efforts by developing plans 
encompassing all three Parts and specifying their interrelationships. In 
general, Part C funds support applied research and developmental stud- 
ies; Part D, demonstrations; and Part I, development of nationally need- 
ed curricula. The usoE staff tries to move useful research products into 
developmental ^nd later into demonstration stages. The staff also strives 
to coordinate vl)cational education R&D work with general educational 
R&D , with R&D |n special education, and with research supported by the 
National Institute of Education (nie). 

Despite those attempts. Parts C, D, and I have not been coordinated 
to produce a well-integrated research and development program. There 
is little evidence that Part D demonstrations are based upon information 
gained from Part C or from products developed under Part I. Instead, 
Part D funds have generally supported career education models. In addi- 
tion, each of the three programs has its own set of priorities that may or 
may not coincide with the priorities of the other two programs. 

To some extent, attempts at coordination are hampered by the legislat- 
ed purposes of the three parts. Part D funds have been used appropriate- 
ly for career education. In general, they cannot be used to support dem- 
onstrations of Part C projects not related to career eiducation. Thus, lack 
of coordination of Parts C, D, and I is a legislative as well as an adminis- 
trative problem. 

Planning and Setting Priorities 

Most of the planning in the Office of Education has been on a year-to- 
year basis although there are some longer-range plans. Lx)ng-range plan- . 
ning of a specific nature is exemplified by the niulti-year commitment of 
the Part I staff to develop curricula in each of the usoE-designated occu- 
pational clusters in the early 1970s. The development of occupational 
cluster curricula was established as a Part I priority in response to pres- 
sures in BOAE but outside the Division. 

ES 70 (Educational System for the 70's) and career education are nota- 
ble examples of attempts to initiate long-range planning, and both are 
also examples of the use of vocational education research funds for pur- 
noses that extend well beyond the goals of the authorizing legislation. 
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Each program was advocated strongly by one administrator and, there- 
fore, was dominant in vocational education r&d for a period of time. 
David S, Bushnell, Director of the Occupational Research and Planning 
Program in the Division of Vocational and Technical Education between 
1965 and 1969, was the major proponent of es,70, es 70 has been closely 
associated with the idea of an ''organic" curriculum that would prepare 
students for a variety of post-high school activities. The organic curricu- 
lum was designed to include both academic and occupational training, 
as well as components of personal development, real work experience, 
and post-high school placement. 

Career education became a high priority with the strong advocacy of 
Sidney P. Mariand, jr.. Commissioner of Education between 1970 and 
1972. The career education program, as originally supported by usoE, 
was transferred to nie in 1972. Poorly defined roles created some difficul- 
ty for NIE and usoe in formulating an r&d plan for career education, nie 
has defined its role with respect to career education as the ''examination 
of the relationship of education and work, and the development of pro- 
grams and products to improve this relationship" (U.S. Department of 
HEW 1975, p. 8). usoE's role has been "assistance to states and local 
education agencies to use, demonstrate and improve the practice of edu- 
cation in relationship to the world of work" (U.S. Department of hew 
1975, p. 8). Not only is it difficult to see the difference between these 
definitions of career education r&d, but also both clearly overlap with 
vocational education r&d. 

The notions of bo'ih ns 70 and career education were dominant (and 
almost exclusive) themes of the federal vocational education r&d pro- 
gram at various times. For example, several of the Section 4(c) projects 
funded during fiscal 1967-69 were directed toward the development of 
the ES 70 program in such areas as career guidance and modern manage- 
ment practices for education. In addition, in fiscail 1972 and 1973, Parts 
C, D, and I were all oriented towards research, development, and dem- 
onstrations that would increase the knowledge base for career education. 

There is no evidence that a national dialogue for planning involving a 
representative, segment of the vocational education community or its 
R&D sub-community occurred during the years 1964 to 1974. A five-year 
or even a two- or three-year vocational education research agenda has 
never been published. While priorities are set yearly for Parts C and I, 
Part D priorities are set e\-ery three years. 

Procedures for setting priorities are similar for all three programs; the 
Part C procedure is described here. The yearly procedure has three phas- 
es. Puring the first phase, which lasts about one year, the Director of the . 
Division of Research and Demonstration meets with branch chiefs and 
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various groups to determine possible priorities. These groups include: 
Tlic Research Committee of the National Advisory Council on Voca- 
tional Education; the State Directors of Vocational Education Research 
Liaison Committee; an ad hoc group of state rcu directors; Curriculum 
Coordinating Center directors; USOE regional personnel; and ad hoc in- 
terest groups. Recommendations from sources such as General Account- 
ing Office reports, usoE program evaluation reports, the Commissioner 
of Education, and influential national leaders are also sought. 

During the second phase, the Deputy Commissioner for the Bureau of 
Occupational and Adult Education makes the final decision regarding 
yearly priorities. Typically, new priorities are favored over the continu- 
ance of old priorities. 

During the third phase, grant announcements and RFPs are prepared, 
moved through the administrative levels, and approved for publication 
in the Federal Register (or in Commerce Business Daily for contract an- 
nouncements). 

The procedure for setting priorities is much more responsive to politi- 
cal pressures than to scientific pressures. It is clear that researchers have 
h ' little representation in Phase I. Strong and Jarosik (1975, p. 6) note 
thai in Phase H "leadership in vocational education tended not to have 
control of how research funds were to be spent," and ofll'ered the example 
of the use of those funds* predominantly for career education. In fiscal 
1972 and 1973 at the direction of Commissioner Marland, the Deputy 
Commissioner specified that vocational monies were to be spent for ca- 
reer education; thus he redirected money that could have been used to 
support program categories that were more specifically implied by the 
vocational education legislation. 

The identification of particular priorities is partly dependent on the 
composition of the ad hoc groups that are convened during Phase I. The 
level of sophistication and particular interests of these groups can and do 
influence the priorities established. Hence, the people who convene them 
influence priorities, and there have been diff'erent conveners over the last 
ten years. Obviously, the use of ad hoc groups that change from year to 
year results in lack of stability in the research program. Concern has also 
been raised about the extensive participation of certain state leaders in 
setting federal priorities. A nationwide program of research should take 
state priorities into account, but it is not clear how great a role the states 
should have in setting federal priorities. Nor is it clear on what basis 
certain state leaders have been invited to participate. 

The lack of stability in priorities has been perceived as detrimental by 
the vocational education researchers interviewed. A concern with short- 
term, product-oriented research, a lack of concern for long-term, pro- 
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gram malic studies, and a lack of emphasis on high-risk efforts that have 
promise of high payoff have characterized Part C priorities. Moreover, 
USOE itself reports that identification of new priorities takes precedence 
over continuation of last year*s priorities, regardless of the probability of 
payoff from further effort toward a goal partly reached during the past 
year. 

Policy Development 

Both understaffing at usoE and frequent administrative shifts may ac- 
count for the lack of long-range planning. Whatever the reason, policy 
decisions have not been derived in a consistent and systematic manner 
and often have been determined externally. The need for quick answers 
to pressing problems, changing goals with each new Commissioner of 
Education, and reaction to political pressures have tended to increase the 
emphasis on targeted, product-oriented priorities. Policy and decision 
making have generally not been influenced by past r&d activities, to 
continue research needed in some areas and to allow for learning from 
past R&D. Stronger and continuing national leadership in policy develop- 
ment is needed. 

Awarding Grants and Contracts 

Announcements The availability of funds and priorities for the federal 
halves of Parts C and D are announced in the Federal Register, which is 
sent to all state departments of education, RCUs, and others and js avail- 
able nationally. Winning applicants are usually awiirded g.rants, unless 
they are profit-making institutions. For Part I projects, requests for pro- 
posals (rfps) for contract a\yards are announced in Commerce Business 
Daily, atlso available to all. 

Some practitioners and researchers believe that the announcement of 
availability of funds in these publications, is, in itself, discriminatory. 
Although the publications are distributed wide?y to departments of edu- 
cation and school districts, they £.10 t;ct available without charge. Small 
private organizations such consulting firms and acafiCmic researchers 
can be especially handicapped Moreover, simply keeping abreast ^f an- 
nouncements and information in Ah*^?-^: prblications is a time-consuming 
task that may b« too expensive for some *«/ould-be applicants. In addi- 
tion, the time for writing applications or propo.sals can be as short as two 
weeks once the Qmiouncement is located aad discussed with collabora- 
tors; however, \\ is generally estimated th&.t it takes four weeks to write a 
proposal and secure necessary local approvals. 
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Nearly all awards are made on a competitive basis and few proposals 
outside the stated priorities are funded. Because priorities change from 
year to year, researchers who want to follow a consistent plan of research 
over several years are likely to be denied an opportunity to compete on 
an equal footing for funds each year, regardless of the quality of their 
past work. 

Announcements requesting grant applications or rfps are essentially 
the only form of dissemination of information about, proposed research 
used by usoe. Grant announcements and rfps specify quite clearly what 
research usoE wants, including goals and methods of accomplishing 
these goals. This procedure requires that usoE have well-defined priori- 
ties, that USOE know exactly how the research should be planned and 
conducted, and, preferably, that the priorities adequately and accurately 
reflect the needs of vocational education. If announcements and RFPs are 
to be well-written and responsive to those needs, good communication 
must exist between usoE and vocational educators. In addition, usoE 
must be adequately staffed with competent vocational researchers or it 
must employ such researchers as consultants. The current limitations on 
salary and expense budgets prevent either of these options from being 
effectively implemented. 

Review Procedures Proposals are reviewed by panels, usually composed 
of five people and chaired by the usoe Part C program chief No more 
than three of the panel members can be federal employees, and they. 
^ must be from outside the Division of Research and Demonstration. 
Hjelm and Boerrigter (1974, p. 43) specify: 

A typical panel will consist of two members being content specialists, one mem- 
ber being a design specialist, one member being an evaluation specialist, and a 
fifth member being an educator or user of the products of the r&d program. 
Attention is given to the geographical spread and the spread by type of institu- 
tion of the non-federal reviewers. The panels are representative in termi of mi- 
nority groups and women. 

These often are not peer reviews (reviews by other vocational education 
researchers) like those often used in social science, physical science, and 
medical research. The review procedure is characterized by a tendency to 
fund projects at the lowest possible cost. 

Award Recipients 

Figure 1 shows the flow of vocational education r&d funds; the dollar 
figures are based on fiscal 1974 data. The agencies and institutions iden- 
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tified in Figure 1 did not necessa'rily receive awards in every year of 
funding. 

Section. 4(c), and Part C The types of institutions that most frequently 
have received grants or contracts have varied from program to program 
and, from time to time. Throughout the Section 4(c) program (up to 
1969), colleges and universities predominated as award recipients, while 
in the Part C program (since 1971), there has been a greater percentage 
of state education agencies (seas) as grantees. In fiscal 1972 and 1973, all 
federal share Part C funds went to seas to support career education and 
experimental, developmental, and demonstration projects. Since the 
state half of Par( C funds is awarded directly to seas, there has been 
extensive state control over research funds; in fiscal 1972 and fiscal 1973, 
SEAS received all Part Cfunds. Research awards are displayed by type of 
recipient for the Section 4(c), federal share Part C, and state share Part C 
programs in Tables 3, 4, and 5. It should be noted that in a few cases 
award recipients may not be the actual researchers because they may 
subcontract the work. 



TABLE 3 Sample of 149 Research Awards by Recipient Institution, 
Section 4 (c) - " 





Percentage of Dollar Awards 
















Private 




I'iscal 


State Education 


Local Education 


Universities or 


' Nonprofit 




Year 


Agencies 


Age ncies 


Colleges 


Institutions 


Total 


1965 


22.8 




65.8 


11.2 


99.8 


1966 


5.7 


0.8 


65.4 


20.1 


100.0 


1967 


10.6 


1.3 


78.6 


9.3 


99.8 


1968 




6.2 


57-7 


33.6 


99.8 


1969 




5.5 


76.6 


9.4 


99.8 


TAB^E 4 Research Awards by Recipient Institution, Federal Part C 




Percentage of Dollar Awards 








Biscal 


State Education 


Local Education 


Universities or 


Private 




Year 


Agencies 


Agencies 


Colleges 


Institutions 


Total 


1971 


1.9 




43.3 


54.8 


1,00.0 


1972 


100.0 








100.0 


1973 


1 00.0 








100.0 


1974 


39.7 


10.9 


27.7 


21.4 


99.7 
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TABLE 5 Research Awards by Recipient Institution, State Part C 

Percentage of Projects • 
Schools and 



State and Local 



f-iscal 


Education 


Universities 


Vo-Tech 






Year 


Af;encies 


or Colleges 


Schools 


Other* 


Total 


1971 


37.6 


45.1 \ 


9.4 


8.0 


100.1 


1972 


38.6 


41.0 


10.2 


10.2 


1 00.0 


1973 


31.7 


44.9 


9:2 


14.2 


100.0 



•Private organizations, individuals, state departments other than education, unknown 
or unclassifiable agencies. 



Fart D Because of the nature af the Part D program, the majority of 
the grants are made to local education agencies. Data collected by De- 
. velopment Associates (1975, p. 142) on the states' share of Part D indi- 
cate that over 80 percent of the grants made during the first three years 
were to local education agencies or schools, 16 percent were to universi- 
ties or colleges, and only three percent were to seas. In many cases, 
federal share Part D funds were awarded to seas (see Table 6). 

Fart I Throughout the Part I program, a significant number of research 
projects have been conducted by private agencies (see Table 7). Many of 
these private agencies are private-for-profit: in fiscal 1972, 23 percent of 
all award recipients were private-for-profit; in fiscal 1973, 20,4 percent; 
and in fiscal 1974, 46.6 percent. (Data on percentage of nonprofit and 
for-profit agencies are not available for fiscal 1970 and 197 1.) Universi- 
ties or colleges also received a substantial portion of Part I funds. 



TABLE 6 Research Awards by Recipient Institution, Federal Pan D 





Percentage of Projects 










State 


Local 










Education 


Education 


Universities 






Funding Round 


Age ncies 


Agencies . 


or Colleges 


Other* 


Total 


1st (fiscal 1970-73) 


23.1 


63.1 


4.6 


9.2 


100.0 


2nd (fiscal 1974-76) 


34.6 


59.6 


5.8 


.0 


100.0 



•Private oreanizations, individuals, state departments other than education, unknown 
or uncla.'^siflable agencies. 
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TABLE 7 Research Awards by Recipient Institutions, Part I 



Pcrcchtagc of Dollar Awards 





State 


Local 








Fiical 


Education 


i:ducation 


. Universities 


Private 




Year 


Agencies 


Agencies 


or Colleges 


Agencies 


Total 


1970 


4.6 


1.4 


33.6 


60.4 


100.0 


1971 


41.8 


2.5 


21.5 


34.2 


100.0 


1972 


21.7 


0.6 


31.3 . 


46.4 


100.0 


1973 


22.7 


3.9 


'47.9 


25.4 


99.9 


1974 


0 


0 


16.0 


84.0 


100.0 



Project pirectors Data on the sex of project directors have been collect- 
ed for Section 4(c), federal Part C, and Part I programs. Tables 8 and 9 
display project awards for these programs by sex of project directors. 
There seems to be a random fluctuation in the rate of female participa- 
tion in the Section 4(c) and Part C programs, with a ten-year average of 
8.3 percent. Under Part 1, female project directors outnumbered males in 
one year (fiscal 1974), possibly due to a large curriculum project awarded 
to the American Home Economics Association (see Table 8). Data on 
the ethnic minority representation among project directors are not avail- 
able. 



TABLE 8 Section 4 (c) and Federal Share Part C Project Directors by Sex 



Fiscal Year 


Males 


I'emales 


Total 


N 


Section 4 (c) 










1965 


79.3% 


. 20.7% 


100.0% 


29 


1966 


88.9 


11.1 


100.0 


45 


1967 


100.0 


0 


100.0 


22 


1968 


95.0 


5.0 


loo.g 


20 


1969 


91.0 


9.0 


100.6 


33 


Part C 






> 0 




1970 


0 


0 


0 


1971 


100.0 


0 


100.0 


33 


1972 


89.9 


10.1 


100.0 


92t 


1973* 










1974 


89.8 


10.2 


100.0 


98$ 



*No data available. 

t3 projects had two co-directors. 

XS projects had two co-directors. 
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TABLE 9 Pan I Project Directors by Sex 



Fiscal Year 


Male 


Female 


Unk nown 


Total 


1972 


25 


1 


2 


28 


1973 


9 


3 


1 


13 . 


1974 


8 


10 


0 


IB 


TOTAL 


42 (71.9%) 


14 (23.7%) 


3 (5.1%) 


59 (100%)ibj, 



NOTE: Only, new starts ure included. 



Monitoring and Evaluating Projects 

X Monitoring ongoing grants and contracts is seen as an important func- 
tion by USOE in an effort to ensure adequate benefits from the expendi- 

, ture of taxpayers' money. In practice,' however, limited numbers of staff 
and insufficient funds for travel prevent usoe from doing as much mbni- 
toring as itHhinks desirable. In general, contracts require closer monitor- 
ing than do grants. 

The Office of Education has only infrequently required evaluation to 
be a component of r&d projects. (In fiscal 1972 and 1973, career educa- 
tion projects funded under Part C were required to have third-party 
evaluations.) Instead, usoe has awarded contracts to three groups— De- 
velopment Associates, Inc., Project Baseline, and this Committee— to 
perform post hoc evaluations of the Parts C, D, and I programs. All three 
groups\^have faced difficulties because evaluation was not given early 
consideration in the design and conduct of most r&d projects. Therefore, 
criteria for "success" and desired outcomes were not defined at the out- 
set of each project, and projects did not keep complete records that 
would allow post hoc measurement and evaluation of outcomes. 

In addition, evaluation of the quality of research performance has 
been missing. There is essentially neither a quality control procedure nor 
a mechanism to ensure that research performers who have done unsatis- 
factory work in the past will not be awarded grants or contracts in the 
future. 



Dissemination * 

The Office of Education relies heavily on written reports for the dissemi- 
nation of R&D products. Abstracts of completed projects are published in 
a periodic usoE report and are sent to Abstracts of Instructional and 
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Research' Materials (aim/arm) and the Science Information Exchange , of 
the Smithsonian Institution. Final reports are sent to Educational Re- 
sources Information Center (eric) and aim/arm. Beginning in 1975, in- 
formation about Part D projects funded by usoe and-NiE will be com- 
piled into an annual report, usoe believes commercial'publication to be 
the most effective means of disseminating curriculum material's and, 
therefore, encourages Part I project;directors to seek such publication. 
However, the curricula developed are often highly specialized and the 
market is considered to be speculative because of the Part I focus on new 
occupations. Therefore, it is often impossible to secure commercial pub- 
lication for these materials. 

The view of dissemination currently most popular among educators 
and social science researchers involves two principles: 

1. To be effective, dissemination must be planned at the outset of a 
' research project. ^ I. 

2. Dissehiination modes must be flexible and must take into account 
the varied needs of the audiences addressed. 

However, usoe does not routinely require that researchers and develop- 
ers carefully consider the appropriate means of disseminating the results 
of R&D efforts before a project is completed. (The National Network for 
Curriculum Coordination and the state rcus were established partly to 
aid in dissemination of R&D projects. These institution^ are discussed 
^ later in this chapter.) 

NATIONAL Vocational education r&d centers 

In 1965, USOE established a series of national and regional r&d institu- 
tions to respond to specific substantive needs in education. -The Division 
of Vocational and Technical Education established institutions related to 
vocational education: an rcu in each state, two national r&d centers, 
and four research development units.' Due to funding reductions, the 
four research development units have been phased out; the other institu- 
tions still exist but receive little funding from usoe. 

The two R&D centers are the Center for Vocational Education at the 
Ohio State University, and the Center for Occupational Education at 
North Carolina State University. Both were funded initially under Sec- 
tion, 4(c) and are now supported by several sources on a project-purchase, 
basis. Each center stresses slightly different aspects of r&d. At present, 
the Ohio Center is much larger than the North Carolina Center. 
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TJ^lNorlh Carolina Center was established in 1965 to serve 14 south- 
ern sl^l^s. Its expressed mission is to provide a national resource for 
policy analysis and development: (I) to inform constituency groups of 
issues that may affect vocational education; (2) to assist federal agencies 
in working with others in policy development; and (3) to respond quickly 
to questions ^f vocational education policy. It also provides technical 
assistance. This Center stresses a multidisciplinary approach to R&D in 
vocational education. ^ 

The mission of the Ohio Center is **to increase the ability of diverse 
agencies, institutions, and organizations to solve educational problems 
relating to individual career planning and preparation." In order to fulfill 
its mission, the Center conducts national programs and projects related 
to six objectives (Council for Educational Development and Research, 
Inc., p. 38): 

1. Generating knowledge through research 

2. Developing educational programs and products 

3. " Evaluating inaividual program needs and outcomes 

4. Installing educational programs and products ; 

5. Operating information systenris and services 

^ 6. Conducting leadership development and training programs. 

Whenever possible, the Center undertakes multi-year endeavors that ad- 
dress national priorities. 

The two national Centers received general institutional support from 
USOE until fiscal 1972 when they were transferred to NIE, which estab- 
lished a project-purchase policy. They currently receive grant and con- 
tract awards for r&d projects from nIe, usoe, seas, local school districts, 
and business and industry. The project-by-project fijnding focuses the 
Centers' efforts on short-term, product-oriented research. 

Both Centers usually receive grants from the Commissioner's shaVe of 
Part C. Although the Centers study problems of regional and national 
scope, these awards are charged against the Ohio and North Carolina 
state allotments of the Commissioner's share; this penalizes both the 
states ^and the Centers and can strain the working relationship between a 
Center arid the state in which it/is located. 

NATIONAL ADVISORY COUNCIL ON VOCATIONAL EDUCATION^' 

The National Advisory Council on Vocational Education (nacve) was 
established by the 1968 Amendments, which authorize funds for its sup- 
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port. It consists of 21 members appointed by the President for terms of 
three years. The legislation specifies that the members be representative 
of labor and management as well as of the general public. They should 
also be familiar with manpower problems and the administration of 
manpower programs, the administration of state and local vocational 
education programs, the training of the handicapped and disadvantaged, 
and post-secondary and adult education, nacve's three tasks, set forth in 
the 1968 Amendments [Sec. 104(a)(2)], are to: 

(A) advise the Commissioner concerning the administration of, prep- 
aration of, general regulations for, and operations of, vocational educa- 
tion programs supported with assistance under this title; 

(B) review the administration and operation of vocational education 
programs under this title, including the effectiveness of such programs in 
meeting the purposes for which they are established and operated, make 
recommendations with respect thereto, and make annual reports of its 
findings and recommendations (iricluding^ecommendations for changes 
in the provisions of this title) to the Secretary for transmittal to the Con- 
gress; and 

(C) conduct independent evaluations of programs carried out under 
this title and publish and distribute the results thereof. 

NACVE has a research committee that commissions research and evalu- 
•ation activities and informs the Council of state vocational education 
research work. The research committee is concerned primarily with the 
second and third tasks listed above. Indicating nacve's interest in r&d. 
in April 1975 Roman Pucinski testified for nacve before the House Sub- 
committee on Elementary, Secondary, and Vocational Education. Piipin- 
ski suggested that a major portion of federal funds for vocational educa- 
tion should be used for supporting innovative programs. 

NATIONAL DISSEMINATION SY.STEMS 

There are two national dissemination systems supported by federal funds 
for vocational education. One is the eric Clearinghouse in Career Edu- 
cation (ERic/ciCE), supported by Central eric at nie: the other is 
AIM/ARM, suppprted by usoe. 

ERIC/CICE was preceded by the eric Clearinghouse in Vocational and 
Technical Education (vt-eric), which was originally established at the 
Ohio Center in 1966. vt-eric was the first clearinghouse in the eric 
system to: (1) develop information analysis products; (2) partition the 
eric file; (3) engage in dissemination activities: (4) computerize its own 
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file; (5) conduct user studies; (6) develop a user training package; and (7) 
develop supplementary abstract publications (aim/arm). The Clearing- 
house benefited from the resources and contacts of the Ohio Center: a 
research library; a large, interdisciplinary staff of vocational education 
R&D specialists; a research program on, dissemination; and links to the 
vocational education community. 

For 1966-68, vt-eric was totally supported with vocational education 
funds authorized by the 1968 Amendments at about $300,000 per year. 
In 1969, Central eric funded the Clearinghouse at reduced levels and, 
for the next few years, funding ranged from $175,000 to $240,000 per 
year. Another cut in funding came when Central eric, which had moved 
to NiE in 1973, merged vt-eric and the Adult and Continuing Education 
eric' Clearinghouse at Syracuse University into a new clearinghouse, 
ERic/ciCE at Northern Illinois University. The contract awarded was for 
about $152,000, far less than the previous year's funding for vt-eric and 
about one half of the combined funding for the two displaced clearing- 
houses. . 

Lacking experience in clearinghouse operation and limited by inade- 
quate funding, s:ric/c:ce could not continue the activities of vt-eric. 
Central eric had drastically curtailed information analysis at clearing- 
houses and discontinued support for local clearinghouse collections. 
ERic/ciCE has not developed a comprehensive information resource sys- 
tem for vocational education even though it has received increased fund- 
ing from NIE in recent years. Its principal emphasis has been upon acqui- 
sitions of research reports, input to Resources in Education, and computer 
searches of the eric files. 

Many potential users of vocational education research products have 
not been totally satisfied with the eric system'. At the summer 1975 meet- 
ing of the six curriculum centers, the two principal complaints against 
ERjc raised by participants were the time delay (about six months) before 
materials submitted to eric are accessible, .'and the screening performed 
by ERIC. In many cases, states want immediate access to all curricula, 
however. Other criticisms of the eric system are that it takes too long for 
requests to be filled and that it is difficult tcj' work with microfiche. 

Because of curtailed eric ciearingliouse services, in 1974 boae decided 
to continue support of aim/arm as a project, aim/arm had been con- 
ceived originally as an ERjc-compatible, supplementary publication pro- , 
viding comprehensive coverage of research and instructional materials / 
for the vocational education audience. With its new project status, / 
AIM/ARM had the additional responsibility bf reporting projects in prog- 
ress, conducting literature searches for boah, supplying bibliographies in 
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support of solicitations for Parts C and D project applications, develop- 
ing interpretative papers, and consulting with dissemination network 
components, 

aim/arm's bimonthly publication, Abstracts oj Instructional and Re- 
search Materials, includes sections entitled: 

instructional Materials Abstracts 
InstructionaK|Vlaterials Subject Index 
Research Materials Abstracts 
Research Materials Subject Index 
Curriculum Development Projects in Progress 
Research Projects in Progress 

The projects-in-progress sections announce current activities funded 
under the 1968 Amendments. In addition to the bimonthly publication, 
aim/arm products include (Magisos 1975): 

Annual indexes to aim/arm 

Computer search tapes, including summaries of at least 16,000 docu- 
ments. 

Microfiche of research documents and products 
Information searches for the sponsor 
Interpretative papers 

Consultation with affiliates in the linked information dissemination 
network 

Repackaged indexes to instructional materials 

Pilot testing of state information dissemination services 

Training workshops for information specialists 

Development of a guide to operating information dissemination sys- 
tems and a guide to existent information resources 

User training and developm^rnt of user training materials. 

To some extent, aim/arm overlaps with hric/cice. Neither is a com- 
plete information and retrieval facility, and both omit certain classes of 
information for reasons other than quality. However, to the extent that 
they are utilized, both systems serve to provide the vocational education 
community with completed and in-progress instructional and research 
materials. The state rcus and regiciial curriculum centers (discussed be- 
low) help to feed information into these systems, encourage more wide- 
spread use, and disseminate microfiche to requesters. 

/ i 



62 ASSESSING VOCATIONAL EDUCATION RESEARCH AND DEVELOPMENT 



REGIONAL ORGANIZATIONS 

NATIONAL NETWORK FOR CURRICULUM COORDINATION 

The National Network for Curriculum Coordination in Vocational and 
Technical Education is made up of six regional centers. It was estab- 
lished by USOE in June 1972 with five centers, after 30 states responded to" 
the initial rfp; in June 1973 two other centers were added. One year later 
the seventh center* was discontinued because of lack of funds, and the 
states were regrouped into six regions more nearly consistent with the ten 
USOE regions. The National Network is supported under Part I; current 
funding is about $40,000 per year per center. 

Originally, there was dual motivation for funding the centers: to im- 
prove the curriculum management capabilities of states and to put cur- 
riculum development and management in the context of career educa- 
tion (Simpson 1975, p. 23). Secondary goals were to improve communi- 
cation among states, especially among neighboring states, to enhance 
coordination in reducing duplication of effort, and to promote coopera- 
lion in developing, validating, evaluating, disseminating, and installing 
curricula. i 

In a statement prepared by usoe for Congressional oversight hearings 
in the spring ol 1974, four primary purposes of the National Neiwork 
were outlined: ; 

• Information sharing: to provide a mechanism for the sharing of 
information on curriculum materials available and under development, 
and for reporting on coordination efforts. 

• Standards: to develop and recommend guidelines for curricula and 
curriculum development with the ultimate goal of increasing the effec- 
tiveness of curriculum materials and enhancing their transportability. 

• Curriculum needs, as a basis for planning: to establish and main- 
tain a system for determining curriculum needs in vocational-technical 
education and reporting conclusions to the field. 

• Coordination: to coordinate activities in curriculum development 
dissemination and utilization with the aim of avoiding unwarranted du- 
plication, enhancing quality of effort, increasing the transportability of 
curriculum materials, and improving the acceptance and use of curricu- 
lum materials. 

Curriculum development is viewed by the- National Network as dis- 
tinct and separate from the rest of r&d. Communication between curric- 
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ulum developers and other researchers is not presently one of the Na- 
tional Network goals. It is important to note that Part C functions over- 
lap considerably with the functions of the National Network centers. 

Each center is presently staffed at a marginal level'and operates as best 
it can under the circumstances, but the centers' efforts are fragmented 
and isolated. The centers duplicate some of the work of eric and 
AIM/ARM, in part because they are not satisfied with these services. Their 
efforts at dissemination are neither as far reaching nor as concentrated as 
nnjght be desired. Their efforts at evaluating their dissemination work 
have been severely hampered and ,sometimes eliminated because of. lack 
of funds. Although the centers are organized along usoE regional lines, 
there is no evidence that geographic differences in curricula are larger 
than the similarities or that the differences that do exist are related to 
variability among usoe regions. Because regions lack the revenue base of 
either local or state government, it is not possible for the centers to ob- 
tain a multiplier effect (matching federal funds with state or local mon- 
ey). There is no apparent reason for regionalization of curriculum devel- 
opment efforts. 



STATE ORGANIZATIONS 

On the state level, institutions involved in the vocational education r&d 
program are concerned primarily with the states\share of Parts Cknd D 
funds. These institutions are the seas (which may be called state educa- 
tion agencies or state departments of education), state departments of 
vocational education, which may or may not be part of the seas, rcus, 
and state advisory councils on vocational educ^ation. 

Vocational education r&d programs vary greatly by state, reflecting 
factors such as the amount of allotments from usoe, the amount of local 
and state matching funds, thS^scope and function of the rcus, the admin- 
istrative structure, and the state's philosophy toward research. State ap- 
propriations to match Parts C and D funds vary widely. In most states. 
Parts C and D funds are not matched equally'by state funds. Small states 
receive very small Parts C and D allotments and seem to be Tinifoimly 
unable to provide sufficient state revenue to have enough money for 
productive vocational education R&D. RCUs, which play a key role in 
states' research programs, usually function as a branch of the state voca- 
tional education department, but in a few states they are located at and 
partly funded by a university or are part of a more general educational 
research operation in the sea. 



73 



\ 



64 ASSFSSING Vr.X:AT:ONAL EDUCATION RESEARCH AND DEVELOPMENT 



RESEARCH COORDINATING 1\>;ITS 

All RGUs adni-jnisier their sutes' share of Part C research funds. Part C 
f?'nds pay up to 75 percent of the costs of operating the rcu and up to 90 
percent of the costs of r&d grants and contracts. Some states support the 
operations of their rcus with state funds and use the Part C funds to 
expand the research effort wiihin the state; other states supply only the 
required matching. 

In addition to managing Part C funds, rcu functioiv. include setting 
research priorities, coordinating state r&d efforts, dtA disseminating r&d 
results and products. In some st;/.tes, the rcus play g large part m con- 
ducting research; in other states, research fundf are awarded to individu- 
als, local educafion agencies, and other 'organizations that conduct Ft:rt 
.C research. In addition, rcus have the potential to form a national net- 
work and link the national state, and local levels of the r&d program. 

Usually, RCUS set research priorities with the approval of the state 
director of vocational education. About half of the ten rcus visited by 
the staff (see Appendix B) claim to consider the federal priorities for the 
Commissioner's projects in setting their own state priorities, but some 
RCU directors believe that federal priorities change too frequently. The 
ten RCUS sampled review their priorities annually, some using a more 
highly refined process than others. Most gather input from teachers and 
state and local administrators and sometimes from lay or labor people. A 
few RCUS use management by objective and therefore stress measurable 
goals and objectives under their priorities. They believe that this is a 
good way of ensuring that state priorities will be addressed and of assess- 
ing the progress of projects done by either the rcu or'other researchers. 

In awarding grants and contracts for Part C, there is considerable 
variability in the use of rfps. (In a few rcus, all research is in-house; 
consequently, they do not use rfps.) Some rcus use some form of an- 
nouncement, usually not as formal or as'structured as the federal rfp. 
There is some hesitancy to use rfps, partly because it is difficult to write 
them. Perhaps even more important is that rfps require researchers to 
prepare lengthy proposals without any guarantee of being funded, and 
this is thought to waste time and energy and to alienate the research 
community. 

All but two of the ten rcus visited fund researcher-initiated proposals, 
although some rcus require that these projects fit within the state priori- 
ties. In one small state where communication between the rcu and re- 
searchers is excellent, all projects result from such proposals. Half of the 
states surveyed use sole-source awards to some extent. Usually they are 
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used for specific ideas judged by the rcu to be important and most 
appropriately handled by one well-known performer. Minigrants (awards 
of approximately 55,000 or less) are frequently awarded to teachers 
or schools because they seem to generate interest and enthusiasm and 
result in substantial informal matching of funds or services from the 
school or locality. However, two states visited do not use minigrants 
because of the high administrative costs involved. 

A common problem faced by the rcus in making awards is the time 
lag between federal appropriations and obligations. According to some 
RCU directors, the uncertainty of funding is a factor that influences the 
use of RFPs, the quality of proposals, and even the quality of projects 
funded. Some rcus have additional uncertainties with regard to state 
support, which further handicaps planning. 

In some states rcus take on additional functions such as training re- 
search personnel, developing a management information system, and 
maintaining a research information clearinghouse. For those rcus with 
little in-house research, typical activities include providing technical as- 
sistance to both producers and consumers of research products and mon- 
itoring ongoing projects. 

In about half the states, rcus also manage projects funded by Part D, 
while in the other states a .separate section within the state department of 
vocational education or the ska administers Part D. While almost all 
SEAS (or rcus within seas) monitor Part D projects, the extent of their 
involvement varies widely. Data collected by Developrfient Associates, 
Inc. (1975) indicated that in 20 states. Part D project sites were visited at 
least monthly: in 22 states, they were visited three or more times per 
year: in seven states, one or two times per year; and in one state, as 
needed. 

The amount of coordination between Parts Cand D is closely related 
to whether or not the two parts are administered by the same agency. 
When Part D is administered by a separate section within the sea and 
not by the rcu, communication problems between the sea and the rcu 
seem frequently to inhibit coordination between the two programs. 
These problems are more acute when the sea staffshows a lack of recep- 
tivity to research. 

Dissemination activities, although variable in type, are carried out to 
some extent by all RCU.s. In all but one of the ten states visited,- final 
reports are required of every project and some of these reports are even- 
tually included in the eriC dissemination system. Most of the rcus have 
ERIC collections with computer retrieval systems, and microfiche is avail- 
able upon request. 

<> 
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. Beyond involvement with eric, rcu dissemination efforts vary consid- 
erably. Dissemination in about half of the stutes visited is geared primar- 
ily to requests for infvirmation. All but three states prepare abstracts of 
projects done in their states in addition to microfiche of final reports. The 
abstracts are usually available upon request and, in sonie states, are dis- 
tributed routinely to a selected audience. In one state, each request for 
information is handled individually and copies of searches are not kept 
for future use. Other states answer some requests with information gath- 
ered for earlier requests. The former method has a personal touch and is 
felt to increase the interest of users; the latter approach is less costly and 
probably less time-consuming. 

Other Rcus take the initiative in circulating information. Several send 
out newsletters or journals that announce the availability of research 
products. Less common vehicles for dissemination are personal contacts 
through workshops, demonstration sites, in-service training, and regional 
centers. It is generally felt that personal contact is the most effective 
dissemination technique, particularly since many potential users are un- 
familiar with research processes, research methodologies, and the use of 
microfiche. Personal contact as a means of dissemination is widely recog- 
nized as very expensive and time-consuming for rcus with relatively 
limited budgets and few staff. 

STATE DEPARTMENTS OF VOCATIONAL EDLC ATION 

Each year the state boards of vocational education, in- conjunction with 
the state advisory councils, develop state plans for voci;ttional education 
as specified in the 1968 Amendments. These plaiis, which identify both 
annual and long-range objectives, focus primarily on instructional pro- 
grams, but they include r&d as well. The plans are submitted to usoE 
and must be approved before a state can receive any money under the 
1968 Amendments. 

In addition to their involvement with the states' share of Parts Cand 
D, state departments of vocational education review and approve pro- 
posals from local education agencies for federal Part C awards. In many 
cases', they even assist in the writing of those proposals. In fiscal 1975, the 
SEAS were also asked to review all proposals submitted from their state 
and to advise usoe of any duplications. Since seas are also eligible to 
receive discretionary awards, there may be a conflict of interest in this 
SEA function. Some seas avoid this problem by choosing not to compete 
for discretionary federal money or by approving all proposals from local 
education agencies that are submitted to them. 
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STATE ADVISORY COUNCILS ON VOCATIONAL EDUCAIION 

The slate advisory councils on vocational education (sacves) are in- 
volved in planning with regard to the states' share of Parts C and D. 
Functioning autonomously, the councils are responsible for evaluating 
the effectiveness of state and local vocational education programs in 
terms of the goals and objectives outlined in the state plans. Recommen- 
dations for change resulting from slate evaluations, contained in annual 
evaluation reports, are sent to the National Advisory Council on Voca- 
tional Education, usoe, the particular state board of vocational educa- 
tion, and other state agencies. The relationships of the sacves to rcus 
differ widely. For example, among the sample of ten states visited, five 
SACVES advise the rcus on research priorities and read and react to re- 
search proposals. In the other five states, the sacves do not do so and are 
not seen by the rcus as being very helpful to them. 

SACVES vary considerably in the size of their budgets. Those sacves 
whose budgets provide for more than a minimal staff are frequently in- 
volved in supporting research to aid their evaluative functions. For ex- 
ample, Illinois had funded 25 projects and Texas had funded 17 projects 
as of June 1975. Most projects were funded by the annual federal allot- 
ment for SACVES, but the availability of state funds for other sacve activi- 
ties frees federal monies for sacve research and evaluative activities. 

DISSEMINATION AND UTILIZATION 

Dissemination and utilization activities have been inadequate for voca- 
tional education r&d to have had measurable impact. After r&d projects 
are completed, final reports are usually prepared (as required by usoe), 
the required number of copies are supplied to usoe, and some distribu- 
tion is made. Reports are then entered in the vocational education in- 
fo alion storage and retrieval systems (uric and aim/arm) and occa- 
sionally reviewed or mentioned in other publications. Although a wide 
variety of techniques for dissemination have been developed, widespread 
dissemination is rare, and little attention has been given to increasing the 
use or adoption of disseminated products. Systematic identification of 
target audiences and packaging of information to meet the needs of 
different audiences have been lacking. Joseph F. Blake ( 1975, p. 25) iden- 
tified some special needs: 

The practitioner groups need information for policy decisions and program de- 
velopment. The researcher-developer group needs information about completed 
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and on-going work to avoid duplication and to benefit by existent knowledge. 
Among these groups the need is for collection and processing of materials acces- 
sible and available on a continuing stable basis. They need a system searchable 
through different levels of sophistication and capable of providing repackaged 
and digestible versions for special target audiences. 

There is little documentation of the utilization of R&D products and 
frequent failure to distinguish between dissemination and utilization. Ex- 
isting documentation of utilization is questionable because it often does 
not really measure the actual adoption of products, but rather the extent 
to which products have been disseminated. For example, some states 
document "utilization" by keeping records of requests for microfiche 
copies of research reports. There is apparent diflficulty in identifying utili- 
zation of the results of information-seeking research as well as utilization 
of the products of development-oriented research. For example, it is dif- 
ficult to determine if the new information gained by a certain research 
activity was considered in the decision-making process that resulted in a 
certain change. 

Blake (1975, p. 52) identifies some of the present problems in improv- 
ing utilization by practitioners: 

• Many practitioners do not understand how r&d can help them deal 
with their daily problems. i 

• Many practitioners need help in identifying, locating, and acquiring 
potentially useful materials. 

• Practitioners need more; information or help to determine the prob- 
able utility, reliability, or validity of r&d products and results. 

• Many practitioners need help in interpreting and applying the 
findings of research reports. 

At present, many administrative agencies have some responsibility for 
dissemination, yet the vast majority of r&d products and results are not 
widely disseminated, and there is no systematic way of determining 
which ones should be disseminated. It is not clear which level of project 
administration (federal, regional, state, or local) has or should have pri- 
mary responsibility for encouraging practitioner utilization of r&d prod- 
ucts and results. 

CONCLUSIONS AND RECOMMENDATIONS 

The following conclusions and recommendations in this section are 
based on the information presented above on the administration of the 
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vocational education R&D program. The Committee recommends 
changes/in only the functions and institutions that are most important to 
the vocational education R&D program. These conclusions and recom- 
mendahons are organized differently from the descriptive material 
above; they are grouped into t^our categories: program structure, pro- 
gram planning and administration, institutions involved in the r&d pro- 
gram, and dissemination and utilization. 

PROGRAM STRUCTURE 

Consolidation and Use of Parts C A and I of the Vocational Education 
R&D Program 

Conclusion The results of research funded under Part C have not been 
fully used as a basis for^developing curriculum products under Part I and 
designing demonstrations under Part D. The lack of coordination and 
the separate setting of priorities among Parts C D, and J have reduced 
the potential for a cumulative impact of the R&D program. In addition, 
there has been some overlap in the activities funded under the separate 
Parts, due in part to legislative mandate. 

At the state level, the points at which the Committee was able to find 
links between Parts C and D were in states in which both programs were 
administered by the rcu. In some states there is virtually no communica- 
tion between the Part C program administered by the RCU and the Part 
D program administered by a division within the sea. 

Recommendation Without reducing the overall appropriation, the Con- 
gress should change existing legislation to provide that the vocational 
education R&D programs be consolidated. In addition, the legislation 
should mandate that not less than 20 percent be allocated for research, 
defined as scientific inquiry designed to generate new knowledge. 

Research, development, demonstration, dissemination, and evaluation 
are related elements of the same R&D system and belong together for 
purposes of administration, coordination, and communication. R&D 
products resulting from research programs should be considered in 
defining some of the priorities for demonstration and curriculum devel- 
opment. Unexpected results from demonstration and curriculum devel- 
opment should be further studied by researchers. Results from research, 
development, and demonstration projects should be considered in setting 
priorities for R&D and should ibe used in the training of vocational educa- 
tion personnel. 
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Administrative Location 

Conclusion Rapid shifts in policies and goals of the vocational educa- 
tion R&D program are due in part to frequently changing administrators. 
Further, the three parts of the r&d program have been shifted and reorga- 
nized within usoE, fragmenting the total effort and, at times, removing 
research from its proximity to the operating unit for vocational educa- 
tion, ^e Bureau of Occupational and Adult Education. 

Recommendation The Committee recommends 16 Congress, the Secre- 
tary of HEW, and the Commissioner of Education that the i^esearch, dem- 
onstration, and curriculum development parts of the vocational educa- 
tion R&D program (Parts C, D, and I) continue to be administratively 
located within the Bureau of Occupational and Adult Educa'tion in usoE 
in order to help ensure stability of planning and toTacilitate coordination 
\ of research with program operations. 

'Commissioner's Share Funds 

Conclusion Because both the Commissioner's (federal) half and stales' 
half of Parts C and D funds are distributed to states on a formula basis, 
SEAS have received a sizable portion of vocational education R&p funds, 
and problems of multi-state or national scope have not received ade- 
quate attention. 

Recof^mendation Congress should designate 50 percent of all vocation- 
al education r&d funds (Parts C, D, and I) as the Commissioner's share, 
reserved for attempts to solve national or multi-state problems; these 
funds should not be subject to state formula allocations. 

Career Education 

Conclusion Career education. has been supported heavily with vocation- 
al education r&d funds since 1971, and activities supported with these 
funds have contributed materially to the development of career educa- 
tion. Career education is a broad concept in which vocational education 
plays an important role. However, there are major conceptual' differences 
in their programs. 

Recommendation The Congress should authorize separate funding for 
career education r&d and vocational education r&d. The Commissioner 
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of Education should articulate or integrate, as appropriate, career educa- 
tion and vocational education R&D. 

PROGRAM PLANNING AND ADMINISTRATION 

Procedures for Setting Priorities 

Conclusion There has not been a long-term, scientifically based sched- 
ule of national priorities for the support of research, demonstration, and 
cumculum development. Only one research topic, guidance and counsel- 
ing, has been included in the list of research (Part C) priorities each year 
of the past decade. Some research topics have appeared as priorities once 
or twice during the past ten years, and some important research topics, 
have never appeared as priorities. During two years, fiscal 1972 and fiscal 
1973, there was only c5^ne designated research priority— career education, 
with . an emphasis on guidance and counseling. There appears to be no 
rational or empirical basis for the inclusion, exclusion, or repetition of 
particular priorities. While priorities are established through suggestions 
from vocational educators and administrators, rarely have researchers 
been adequately represented in the process of setting priorities. Those 
involved in vocational education R&D have not been informed about the 
process and have no formal means for participating in it. 

State priorities for research and related activities are often determined 
administratively without any involvement of research producers and 
consumers in the state. In many states there appears to be no formal 
procedure for setting priorities. 

Recommendation A systematic, open, cumulative, and data-based pro- 
cess should be initiated by the Commissipner of Education for identify- 
ing* national priorities for vocational education R&D and by the state 
directors of vocational education for identifying state priorities. The pro- 
cess should involve advisory groups at both national and state jjCyels tha\ 
represent the clients of vocational education, including students; employ- 
ees and employers, and professionals in vocational education. More in- 
put from researchers is necessary to determine what R&D is feasible from 
a scientific viewpoint and to identify the most appropriate mode of y/ork- 
ing on each problem and the most productive sequence of working on' 
different "problenis. Terms of advisory group members should, be suf- 
ficiently long and overlapping; to allow setting priorities that are long- 
term and programmatic as well as those that address problems requiring 
a more immediate response. Priorities should be reviewed and updated 
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annually. The process should be well-fju6licized so that those who want 
to participate can do so. The process s^hould also use the cumulative data 
base developed by management information systems. 

Administration of Awards 

Conclusion The availability of the Commissioners share of federal 
fuiids for vocational education r&d projects has been advertised as rfps 
in the Commerce Business Daily and as grant competition announce- 
ments in the Federal Register, Awards for Parts C and D are distributed 
according tqf a state formula, and applications (or proposals) are subject ^ 
to veto by state directors of vocational education if the directors judge 
them to be duplicative.^Usihg only rfps, especially when further restric- 
tions are imposed by state veto, does not adequately allow or encourage 
researchers with an exceptional idea outside .the announced fesealrch 
•priority areas to submit proposals or applications. 

In some states, the states* share of federal funds for r&d is allocated 
primarily for the study of state administrative problems. In other states, 
. priorities for allocating resources are determined solely by administrators 
(state direqtors of vocational education or rcu directors). Some states 
^announce priorities and issue rfps or announcements of award competi- 
[tions. Only a few states follow procuretnent policies that permit open 
(Competition in response to state priorities determined through open pro- 
cedures. ' . , 

Recommendation A broad mix of announcements and funding proce- 
dures should 'be used by the Commissioner of Education and, the state 
directors of vocational education. The appropriateness of contracts, 
grants, or sole-source funding will vary with the nature of each project 
and the general availability and interest of competent researchers. It 
should be recognized that field-initiated applications and proposals that 
are not in response to contract or grant announcements also have advan- 
tages. In the interest of encouraging innovation, a portion of r&d funds 
should be reserved for good proposals or applications that do not ad- 
dress federal or state priorities or are not in response to rfps. 

In order to accommodate the variable capacity of contenders to pro- 
duce proposals rapidly, usOE and the states should allow potenti.'il appli- 
cants at least two months to respond to the announcements of all compe- 
titions for federal research funds. A pre-announcement of the date on 
which a grant competition is to appear may be useful. usOE and the 
states should also experiment with a two-phase announcement process: 
in the first phase, an initial announcement would solicit project prospecti; 
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in the second phase, those applicants whose projects are considered 
promising would be encouraged to develop full proposals. 

The Role of Women and Minorities in Research Activity 

Conclusion Women have been involved as^ researchers in vocational 
education far less often than men. There are virtually no data on the 
involvement of minorities in r&d projects. Further, vocational education 
researchers have not taken into account research on perceptions held_by 
different groups within society concerning' the acquisition of desirable 
skills, role performance, and attitudes toward time and work. 

Recommendation The Commissioner of Education and state directors 
of vocational education should ensure that researchers and administra- 
tors representative of population subgroups (women and minorities) are 
involved in the r&d program. All rfps an'd announcements of research 
opportunities should state that women and minorities as well as others 
are encouraged to apply. In addition, for the purpose of ascertaining 
trends in the participation of various population subgroups in vocational 
education research activities, the Commissioner and the state directors 
should keep annual status reports on the percentage of r&d project di- 
rectors who are members of various population subgroups. The Commis- 
sioner should also encourage the involvement of researchers from popu- 
lation subgroups in R&D concerning those subgroups. Funds for training 
personnel should be used to build the r&d capacity of these groups. It 
should be determined how data on cultural differences can be used to 
create opportunities for equitable access to vocational programs and 
jobs. 

Management Information System 

Conclusion USOE does not have an efficient system for collecting and 
recording information on many aspects of the vocational education R&D 
program. Little information on research performers is collected; records 
of project impact are not usually kept; and those project evaluations that 
have beer done have rarely been analyzed in depth and used to improve 
prog: ims. Moreover, as in any research program, not all project di- 
rectors submit interim and final reports. Therefore, it is extremely dif- 
ficult to measure the impact that r&d projects have had on vocational 
education. There is essentially no evidence of quality control of research 
performance to ensure that the quality of past work of researchers affects 
the probability that j wiP receive awards in the future. There is no 
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system fbrcollecfing infurn.auon oi^' the research needs: of vocpiionai 
-educators or on unnecessary dupiic^ition of research proji^cts. Ir vidf- 
tion, there is no way to determine whether. or nor. isolated charge} of 
duplication of effort are arcurace. 

Some states rely heavily on management information systems >•;• 
ethers lack systems for information collection and use. The extensiveriess 
of nroniibring and evaluating activities and the extent to which systems 
are dynamic and interactive aho vary greatly among states, j 

Recommendation The Commissioner of Education shouldj provide, fc" 
the development and operation of a national management linformatic 
system for vocational edur.'\tion r&d. Such a system should include 
for monitoring and evaluating projects, measuring dissemination ar 
utilization of project results, keeping track of the quality jof researcli 
performance, and maintaining records of the characteristic's of the reA 
search prograru (incriuding dat or * ' nrch performers, institutional \ 
affiliation of award recipients, tv piojects funded, and amount ^ 
awarded to each stale). Analysis «;ita would help USOE identify 

factors thai are critical m deterrni,j.. , . • success or usefulness of r&d 
projects. The development of state management information systems 
should aiso be encouraged by state directors of vocational education. 

An important funcaon of a management information system for r&d 
would be to provide a '-ystematic means for collecting information on the 
needs of vocational educators. At the same time, topics that have re- 
ceived repealed and diipiicative attention could be identified and unnec- 
essary duplication eliminated. 

iNsrr> UTioNs 

Naiiofial Vocational Education RdD Centers 

Conclusion The national vocational education r&d centers serve useful 
and essential fiinctions. The centers ha^e increased the research capabili- 
ty of vocaiiontii education and have studied topics of national impor- 
tance They have considerable potential for filling the present need for 
national leadership In policy making for vocational education r&d. 
However, th*,* centers must seek support by pursuing contracts and 
grants on a project-purchase basis from diverse agencies. Thus, only in 
scattered instances have they been able to devote themselves to long- 
range vocational education projects of national importance. In addition, 
the centers are forced to compete with agencies within their states for 
Part C funds, TTiis has tended to impair the relationship between the 
centers and the vocational education conimunities withm their states. 
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Recommendation Congress, the Secretary of hew, and the CommLssio' • 
er of Education should ensure that there is at least one adequately fun 

national vocational educatic/.: r&d center. The national r&d effort in 
V Ideational education needs continuity, from setting priorities to the use 
of validated results. The efforts would be enhanced by comprehensive- 
ness that goes beyond that normally achieved in single, unrelated proj- 
ects. Variety and quality of work would be increased through the opera- 
tion of two or three national r&d centers. The center(.s) should receive 
support from federal vocational education funds, including adequate re- 
sources for activities initiated by the centers. 

The center(s) should address priorities that meet primarily national or 
multi-state (as opposed to state or local) needs and should build R&D 
capacity. However, they should also be free to contract with^any state or 
locality to provide needed services so long as these do not interfere with 
the prirrrary task of meeting national and multi-state needs. The center(s) 
should aid in planning and policy development for the national voca- 
tional education R&D system, including providing usoE and states with 
data needed for planning, usoe should view the center(s) as the appropri- 
ate place for conducf.ln^ high-risk research where the payoff may be 
high. The center(s) should communicate with the research operations 
within each state: they should assist in dissemination of research prod- 
ucts and in the training necessary for carrying out r&d activities. To 
increase the relevance of research, practitioners (teachers, counselors, 
etc.) should be involved in center operations, including planning, policy 
decisions, and improving dissemincition capacity. There should be fund- 
ing to allow practitioners to work with center(s), through means such as 
grants and summer institutes. 



Curricu'um Centers 

Conclusion The National l^'^i^twork for 'Curriculum Coordination in Vo- 
cational and Technical Education funded by Part I has received less.than 
adequate federal financial support for its intended activities. It is a re- 
gional effort, but the only regional financial support thp.r is avaiL.ble 
comes through voluntary cooperation of groups of states; in only rare 
/ instances have groups of states contributed the necessary finances, Con- 
{ sequently. the efforts of the network have beew fragmented and ineffec- 
( tive. Furiner, the centers are duplicatin*: • of the functions of a com- 
' prehensive information system as well . • -:me Part C functions usually 
performed by Rcus. There is no apparent reason for separating the coor- 
dination and dissemination of curriculum development products from 
th-l of other r&d products. 
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Recommendation The Commissioner of Education should require that 
the following curriculum coordination functions be performed: (I) iden- 
tifying common curriculum needs among states; (2) encouraging local 
practitioners to become involved in curriculum development; (3) feeding 
curriculum information for the tes into eric and aim/arm; (4) per- 
forming curriculum searches of these systems for vocational educators; 
(5) improving techniques for curriculum development, and (6) discourag- 
ing unnecessary duplication in curriculum development. Since curricu- 
lum development is very expensive, eliminating unnecessary duplication 
is economically desirable. 

If adequate funding for the Curriculum Center Network cannot be 
provided, the Network should be disbanded and the functions should be 
assumed by the national vocational education R£ 'j center(s). The r&d 
center(s) should receive additional funding, which would probably be 
less than that needed to support separate institutions partly because of 
economies of scale. 

Research Coordinating Units " 

Conclusion Reus vary widely in organization, function, and effective- 
ness. All Rcus^have attempted to stimulate state and local interest in the 
R^:D process and disseminate information on r&d products, thereby legi- 
timizing K&D within the states. However, many states have had difficulty 
in disseminating research results and products and in promoting their 
utilization. In addition, some states deliberately separate the administra- 
tion of research from that of development and demonstration. In small 
states, RCUs have not had enough money to support research, a full-time 
director, and clerical services. In some sta'tes, Part C funds are being 
used for. purposes that should be supported by program operating funds 
.(Part B), for example, operation of state nianagement information sys- 
tems and routine program evaluation (although development of manage- 
ment information systems and plans for evaluation are legitimate r&d 
functions). 

Recommendation Congress and the Commissioner of Education should 
ensure that the Office of Education allocate funds specifically for the 
ROUS based on a periodic, evaluation of each rcu's activities. States that 
have effective Reus should receive a minimum allocation (approximately 
$25,000) plus a g^opulation-determined amount, even if this necessitates a 
cutback in the present funding levels of the larger states. The states 
should be encouraged to provide additional funding from program rnon- - 
ies (Part B) and from other state sources. One way of doing this would 
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be to require that federal research funcJs for RCus be matched (perhaps at 
a 25-percent rale) from federal funtli^ for program operation or slate 
sources. 

The Office of Education should pn^^ide guidelines and organizational 
support for the management of r&P us and for the establishment of 
cooperative activiues among Reus. 

RCUS should be responsible for n^jinagf^ment of demonstra- 

tions (Part D), research (Part C), iind v:urricuii:m c!cvelopment projects 
(Part I). They should be required ' 'develop a process for defining re- 
search priorities for their states an^ providing input for setting priorities 
at the federal level, rcus should alsCkJ?^ concenied with '^e validation of 
R&D products before these produces widely disseminated or imple- 
mented in the state. RCUs should that intensive evaluation be 
built into a sample of their ^ jarch projects, rcus should be required to 
continue dissemination and utilization efforts in order to develop their 
capacities to serve as brokers of 9^^^- rCUS should cooperate with the 
national r&d centers and with oth^r n^^iional dissemination efforts. RCUs 
should both provide intellectual lea^ersj^jp for research and involve local 
practitioners in research in order to facilitate the utilization of R&D re- 
sults in vocational education. 

DlSSEMlNATlOIsl aND UT1L1Z/\TI0N 

The Committee views dissemination ^^d utilization as extremely impor- 
tarjil aspects of the vocational educ^^ti^tx g&D program. Both are essential 
in moving, R&D products into oper^^^i^^S vocational education programs. 
Three components of the dissemiiiatioi^ and utilization process are dis- 
cussed below in separate conclusi^^n^ ^nd recommendations: informa- 
tion collection and retrieval, infornuition analysis, and utilization. Dis- 
semination is discussed in conjunction ^vith each of these components. 

Information Collection and Retrievdi 

Conclusion Research- reports and ^^^^r products of vocational educa- 
tion R&D have been made accessible the NiH-sponsored eric s^ ' Cm, 
supplemented by the usoH-spons^*'^*"^ aIm/arm project. However, the 
development of a comprehensive inf^^'^ation resource system linked lo a 
dissemination network has not received adequate support from Central 
ERIC. The work of KRic/ciCE and ajM/\RM have not been sufficiently 
integrated ^^^nd, lakeii together, have ot\jy partly met the needs of voca- 
tional euuCir'^^un ;jersOnnel A jdiovisi^^l materials are one of the major 
omissions fr.'::- ,?;^»sting retrieval syS^ern^ 
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R.^ronwiendarion The Secretary of hew and the Commissioner of Edu- 
cation 5^hv. i»Id ensure that vocational education has a comprehensive in- 
fcik-n-ition resource system linked -o a dissemination network serving 
practidoners. The vocational and technical education portion of 
/C(L7 should be administered separately by usoe's Bureau of Occu- 
^al and Adult Education until- C -ntral eric is able to support a 
i'lehensive system for vocational education. The system should be 
ated by an organization that has demonstrated ability to coordinate 
.ergent activities and develop strong links with practitioners. An ade- 
quately funded clearinghouse for vocationar and technical education 
should include ajm/arm and articulate with other vocational education 
R&D activities. 

Every vocational education r&d project should be required to submit 
its reports and products to er'-'s Resources in Education and Current 
Index tp Journals in Education, to aim/arm, and to vocational education 
research libraries, where they will be available for selection, adaption, 
adaptation, and installation by us^rs through' the linking dissciiiination 
network. In audition, selected'projects should be required and funded to 
widely disseminate their own r&d products. There should be an intensive 
program of activities to help rcus intheir-dissemination role in a com- 
prehensive dissemination network. The eric and aim/arm systems 
should cooperate with other agencies to maintain syste . compatibility 
and avoid duplication of effort. The vocational education information 
resource system should cooperate with Central eric, eric contractors, 
other information systems, and the vocational education community to 
overcome the persistent technical problems related to copyrighted, non- 
print, and po >r print materials. Some provision shoi M also be made for 
establishing a system for maintaining and disseminaimg audiovisual ma- 
terials. • 

. i ■ 

Information Analysis 

Conclusion Many vocational educators are either unaware of r&d re- 
sults and products or are unable to understand (and subsequently use) 
them. Simply mailing out copies of reports is not always adequate. In- 
formation synthesis and analysis have not been given adequate or long- 
iasting support by usoe. Various kinds of information analysis are often 
desirable: summary and synthesis of research on the same or related 
topics, and research analyses performed for different purposes or differ- 
ent audiences. Nonprint media are often effective but not often used. 
Personal contacts between the res^^arch and practice communities at 
workshops and ' monstration t.les have been more successful than writ- 
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ten fonv.: \ amunication in disseminating r&d but have been used 
much less i- ^' .ently. 

Recommendation The Commissioner of Education should establish an 
information analysis program \o transform information on critical prob- 
lems into appealing, new forms targeted to diverse user groups. These 
new products should include interpretations .of reseafc^i and commis- 
sioned analyses of research. Specia) collecriions of information on topics 
such as vocational education consultants, nonprint media, legislation, 
and exemplary programs should be developed. 

The Commissioner of Education should fund studies of the needs of 
users of R&b to determine the most effective methods of dissemination 
and the most appropriate forms of information analysis for different 
situations and different users. New products and dissemination strategies 
based on the results of user studies should be developed and implement- 
ed. / 

Uiilizatkin 

Conclusion in general, neither the federal sponsors (in requests for pro- 
posals and grant announcements) nor researchers have planned for ade- 
quate dissemination and utilization activities. There has been little effort 
to increase user receptivity to r&d outcomes, to provide technical assis- 
tance in utilization, or to document utihzation. Dissemination and utiliza- 
i\on have not been included among federal priorities for vocational edu- 
cation R«&D. 

Recommendation USOE must assume responsibility for ensuring that 
RAD results ru'id products are disseminated aad utilized and should desig- 
nate a signiijcani proportion of federal r&d funds for these activities. 
RCUs should be funded and encouraged to serve as state-level links in the 
dissemination network a.nd to provide practitioners with opportunities 
for involvemei t in ^ht: r ■ process.' The national center(s) should be 
made responi-:' le for a:v:es,- 'ng r&d outcomes, for creating user aware- 
ness of promis-'ii.i. \n ,igv .\o\r. aiVc for assisting users with application of 
suitable R& J* ouL":o. *v riodically, selected seas, local education agen- 
cies, unive. ^ iW ■ f i'.^' :sio?.^a! organizations should be funded to dem- 
onstrate dissc.». acl'on siraiegics, to assist with dissemination and utili- 
zation of specific products, and to conduct research on dissemination' 
?jid utilization. User training pi ogran" : involving R&d personnel, admin- 
if:tr:Mi>rs, and educators designed to improve the fiow of information from 
th^ rrso'.-rcc system to the practice community should be conducted. 
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Appendix A 

Review of R&D 
in Major 
Priority Areas 



The members and staff of the Comirdllee on Vocational Education Re- 
search and Development reviewed literature on nine major research top- 
ics: 

Career development and guidance 
Studc:>ts wi'th special needs 

Characterisiic oT students » 

Teacher education 

Instructional techniques 

Curriculum development 

Labor market supply and demand information 

Administration of vocational education 

Evaluation of vocational education programs 

Sources included 15 papers commissioned by the Committee (see Appen- ' 
dix B) on various aspects of vocational educatiop r&d; the review and 
synthesis monograph series published by the Ohio State University Cen- 
ter for Vocatiorifal Education; and numerous reports recommended by 
members of the Committee and others working in vocational education. 

These resources dia not provide a complete review of vocational edu- 
cation R&D over the last ten years. The Ohio State monograph series, 
which - 3 intended to review and synthesize the literature in several ^cate- 
gories within vocational education, is necessarily selective in the findings 
reported and the topics reviewed. Further, the less recent monographs oh 
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the series, written between 1966 and 1970, do not contain current in- 
formation. Unfortunately for this Committee as well as for the progress 
of vocational education r&d, the monograph series was largely discon-^ 
tinued after 197? when funding was withdrawn. 

As' noted in Chapter 4, the*Committee was unable to review projects 
funded only under the 1963 Act and the 1968 Amendments because 
many project reports do not exist and because it was impossible in many 
cases to identify a project's funding source. Therefore, this review covers 
R&D projects pertinent to vocational education, whether or not they were 
funded under the vocational education r&d legislation. 

CAREER DEVELOPMENT AND GUIDANCE 

One line of research in career development. dates back to Super's longitu- 
dinal Career Pattern Study of the late 1950s, which became the founda- 
non for the self-concept approach to career decision making (Herr 1975). 
Super's self-concept theory suggests that people choose careers in which 
they can implement their self-concepts. Students' self-concepts are hy- 
pothesized to be similar to their descriptions of people in the occupations 
they feel they will eventually enter. 

Hypotheses derived from Super's theory have been tested by numer- 
ous educatiokial and other researchers. For example, Ziegler gathered 
data from 428 male college students on self-descriptions, preferred occu- 
pations, and probable occupations' (Mitchell et al. 1975). Study results 
indicate that students saw themselves as being more like people in ca- 
reers they w^;nted to enter than like people in jobs they dislilcr;'! finding 
that supports Super's sel^-concept theory. However, the prer'v.e events 
and experiences that create various self-concepts have not beer- specified. 

Like Super, many theorists have been concerned with occupational 
selection as an expression of personality. For exariipie, Holland's model 
of vocational choice behavior includes a six-category typology of person- 
ality and predicts that- individuals will choose occupations in categories 
consistent with their personality types (Mitchell et al. 19X5). Empirical 
studies exploring this theory have yielded inconclusive results, partly due 
to the difficulty of assigning students lo personality types. What causes 
people to prefer occupations in one or more of these six categories re- 
mains to be discovered. " ^ 

Krumboltz (1975) has advanced a social learning theory approach 
specifying factors that influence educational or occupational preferences. 
Mitchell's review confirms several propositions of the theory, showing 
that educational or occupational preferences are related to (I) positive 
reinforcement (such as successful performance iji a course), (2) reinforce- 
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ment by a valued person (such as a parent or a favorite teacher) who 
advocates entry into ourse or occupation, or (3) exposure to **positive 
words and images associated with the course, occupation, or field of 
work" (Mitchell 1975, p. 42). Other studies confirm that an individual is 
more likely to enroll in a course or seek employment in an occupation if 
he or she has recently expressed a preference for that course or occupa- 
tion, if opportunities exist for participation or employment, or if he or 
she has learned skills that match the occupational requirement (Mitchell 
etal. 1975). 

Still other studies, reviewed by Homer, Buterbaugh, and Carefoot 
(Mitchell et al. 1975), show that occupational choice is influenced by the 
occupation of the father, attitudes of parents toward education, and the 
education of parents. They note that on-the-job experience influences 
decision making; rural students are more concerned with learning a 
specific vocation than are urban students; and farming as an occupation 
is more often transmitted from father to son than are other occupations. 
Many theories of occupativ .al decision making have one common impli- 
cation for vocational guidance: guidance should not merely react to a 
problem or concern an immediate choice, but should teachi decisiori- 
making skills appropriate to vocational choices. However, evidence that 
teaching such skills results in better decisions does not exist. Although a 
large body of empirical data exists^on certain aspects of vocational deci- 
sion making, knowledge is sketchy and the studies cannot be compared 
easily with one another (Mitchell et al. 1975). ^ 

With the growth of career development theoi^^e elements and ob- 
jectives of vocational guidance have received more attention. Fopexam- 
ple, Martin developed a conceptual model for the design of guidance 
materials fof non-college-bound and culturally : disadvantaged young 
people (Tennyson 1968). Krumboltz and his colleagues evaluated the 
success of specific guidance techniques in effecting desired* changes in 
vocational behavior. They foi.md that "verbal reinforcement of informa- 
tion seeking responses during the (counseling) interview resulted in 
greater exploration of relevant occupatiorial "and educational informa- 
tion outsidepf the counseling situation'* (p. 360). 

Other research has attempted to identify effective guidance techniques, 
such as counseling skills required to accomplish specific goals. Counsel- 
ing skills identified by researchers at Michigan State and Stanford Uni- 
versities are related to contact, postural* position, reflection of feeling, 
and summarization of feeling (Heir 1975). The trend toward creating 
behavioral objectives for those receiving vocational guidance has accel- 
erated since the career education movement began in 1971. Researchers 
at the Center for Vocational Ec'ucation in Ohio created a ten-phase mod- 



86 Appendix A 

el for vocational guidance programming that includes the translation of 
goals into student behavioral objectives (Herr 1975). 

Measurement devices have been developed to gauge success in attain- 
ing the specific behavioral goals of career development. For example, 
Crites developed the Career Maturity Inventory, composed of an atti- 
tude scale and a competence test intended to measure work orientation, 
independence in- decision making, self-appraisal, and occupational in- 
formation (Herr 1975). 

Several computer-based guidance systems have been developed, such 
as the Pennsylvania State University Computer-Assisted Career Explora- 
tion System. These systems attempt to provide accurate and complete 
occupational information for rational decision making by students. In- 
formation is provided in multimedia forms, such as slides and computer 
printouts. Some systems even provide students with training in decision 
making (Herr 1975). 

Many of these programmatic approaches to vocational guidance focus 
on preparing students to'-deal with <he process of career c^ecision making 
rather than with the actual career choice (Herr 1.975). Vocationai educa- 
tion needs more knowledge )f liow and wliy people choose and change 
careers. 



student:, with special needs 

In accordance with the legislativr intent that RcScD be directed toward the 
problems of the disadvantaged and the handicapped, researchers have 
investigated the characteristics and problems of groups Mth special 
needs,, and programs have been developed to serve those groups. Also, 
recent legal and so'cial pressures have prompted r&d to study women in 
vocational education. 

( , ■> 

. .. ... ..^ 

Women v. ' ^ 

An Ohio State research- synthesis monograph on women in the world of 
_ work describes studies that jiave assessed the training needs of women, 
patterns of expected growth in job openings for wpmen, and influences 
on the vocational choices of women. One study identified 'be attitudes jof 
women toward careers and marriage by analyzing data on .i career histo- 
ry sheetSand a set of attitudinal scales for a sample of 1,237 girls and 
women. Attitudes affecting life-style .included: "(0 a woman's impres- 
sion of male^s reaction to the use of her intelligence; (2) struggle over the 
possible position of dominance of men at work and the place of women 
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at home; and (3) conflict between family and work demands upon the 
time of wife and mother" (Kievit 1972, p. 63). 

Although there are some studies on women in vocational education, 
Roby (1975) notes that there has been relatively little vocational educa- 
tion R&D funding deyoled to the needs of women, particularly since more 
than half the slildents in vocational education programs are women. For 
example, in ftspal 1974 only one out of 93 federally funded and adminis- 
tered Part C projects pertained directly to women. In the same year, no 
projects specifically on women were funded under Part 1 or Part D. 
Furthermore, in 1975 only three of the 50 slate departments of education 
were sponsoring research on the needs of girls and women. 



The Disadvantaged 

R&D concerned with problems' of the disadvantaged became a uson 
priority in. fiscal 1971, although legislation first emphasized the problems 
of the disadvantaged in 1963. Many studies concern the assessment of 
needs, idenlifioation of special problems, or development of programs to 
meet special r.ekls. The Ohio State monograph synthesizin;; research on 
the urban disadvantaged acknowledges that research has lacked exten- 
"siveness .and quality. Topics needing more attention include public 
school vocational education programs, national surveys, icher prepa- 
ration, student follow-up studies, arj'd structured evalu:: i (Ldckette 
arid Davenport 1971). In addition, "among the studies related to voca- 
tional education jvhich exist, there i$ considerable duplication of data" 
(p. 34). 

However, some significant findings on the disadvantaged have 
emerged. For'example, it has been shown that the involvement of the 
"community in vocational education 'programs for the disadvantaged in- 
creases the completion and placement rates of both in-school and out-of- 
school trainees (Lockette and Davenport 1971). Also, research on out-of- 
^chool vocational education programs for the urban disadvantaged has 
shown them to be more successful than in-school programs in placing 
graduates in jobs. • \ 

Some research has been done tcj determine the vocational education 
needs and characteristics of young people in rural areas and to compare 
these with the needs and characteriistics of ^rban youths. For example, 
Boykin's findings support the generalization that the educational and 
occupational aspirations of urban youths are higher than those of young 
people in rural areas (Griessman ^nd Densley 1969). It has also been 
shown that most rural students ultiinately look for urban jobs. This sug- 
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gests that local occupational surveys cannot accurately determine what 
occupations should be taught in each school. 

/' 

Adults 

Research related to adult students has been directed towards identifying 
effective techniques for teaching adults. For example, one study reports 
that programmed learning was more successful than the lecture-discus- 
sion method in terms of total knowledge gained in adult vocational agri- 
culture departments in five Northeastern states (Adams 1972, p. 38). 




Ethnic. Minorities 

Phyllis Hamilton's report nv75) o-i vocational education R&D on the 
needs of ethnic minorities stiUe^ that little researcjh has been aimed to-, 
ward minority student needs, in part because administrators of vocation- 
al education r&d apparcinly hope that special (needs could be met 
through general research on vocational education! From fiscal 1964 to 
fiscal 1969, about eight percent of Section 4(c) funding was devoted to 
ethnic minority needs, and about five percent each year of Part C fund- 
ing was concern e^d with the problems of minority students. 

A few studies focusing on the needs of ethnic minorities have received 
much attention. For example, a study by Wilford Wilms Hov^^s (hat nei- 
ther public aor v^cjational education has been successful in hL;^,,ng mi- 
nority students, overcome barriers of class and incoir ^Hamilton 1975). 
However, whether^or not vocational education can ^ly be expect- - 

ed to overcome these barriers has received, littr" • Hamilton 

draws several conclusions on the basis of her rev j conducted 

under the 1963 and 196§ legislation (pp. 40-49) 

1. The smail amount of vocational educal'^^r- rc^crt';'_:2 thai has been 
conducted for ethnic minority student needs h:i> been underutilized in 
program development. 

2. The negative image of vocational e^ii; rio; held by minority' 
groups has been reinforced by its use of lab; ^ /.ch as "disadvantaged." 

3. Much of vocational education researc;- has been based on stereo- 
types of the "culturally disadvantaged." Few have tried to identify posi- 
tive attributes. 

4. Specific skill training has been a major emphasis of vocational edu- . 
cation research for ethnic minorities although remediail basic academic 
training was a minor emphasis. 
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5. Part D exemplary projects and fiscal 1974 projects show increased 
emphasis on more relevant career guidance for minority students. " 

6. A negative self-concept was seen as the biggest block to motivation^ 
for ethnic minorities: use of peer counselors was found to raise self-im- 
age. 

7. Staff attitudes, expectations, and behaviors are critical variables in 
providing effective vocational training for minority students. 

8. No research on recruitment was conducted, but use of classroom 
para-professionals was a major theme of training activities. 

9. Little research on ethnic minorities has concentrated on improving 
external linkages with business and industry. 

10. There is an emerging bicultural emphasis in vocational education 
research activity on ethnic minority needs. 

CHARACTERISTICS OF STUDENTS 

Several research efforts describe the personal and social characteristics of 
vocational education students. Although a. 1970 usoe study (Lecht 1972) 
reports that vocational students resemble the general student population, 
most studies indicate that vocational students have lower socioeconomic 
status and less academic ability, than other students. The usoe report, 
although fairly recent and involving 29,000 secondary school students, 
contains data inconsistencies noted by Lecht, and its conclusions are 
therefore questionable. 

Other national studies— Project talent and Somer's 1966 national 
survey of 1,500 students— indicate that the heads of families of vocation- 
al students have had less ediication than family heads of high school 
students in the academic curriculum (Lecht 1972). Project Talent data 
also indicate that vocational students have less academic ability than 
other. studentsi as measured by tests of verbal knowledge, verbal reason- 
ing, mathematical aptitude; general "knowledge," and similar indices. In 
addition, vocational students differ on the average from the general stu- 
dent population in their socioeconomic background as measured by oc- 
cupation, income, or education of family heads (Evans and Galloway 
1973). The degree to which these differences are due to student choice or 
to assignment by the school (tracking) has not been determined (Bowen 
1975). 

A 1969 USDE survey and a University of Wisconsin national survey in 
1966 collected data on the racial composition of vocational education 
programs. The usoe survey found that 20 percent of secondary school 
vocatit)nal students were from minority groups, while the earlier Wiscon- 
sin study indicated that eight percent were from minority groups; this 
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suggests that vocational education may have served more minority stu- 
dents in 1969 than in 1966 (Lecht 1972). Official reports indicate that the 
proportion of '^disadvantaged*' students in vocational education pro- 
grams has increased since 1966. but the data are not conclusive, and the 
extent to which "disadvantaged'' students are from minority groups is 
not clear. Changes in definition and classification and lack of informa- 
tion on students' backgrounds collected by local school districts have 
limited the collection of national data on racial and other characteristics 
of students. 

In addition to national surveys, stat^ research projects have attempted 
to obtain information on the age, sex, socioeconomic background, ar.d 
academic ability of vocational education students. Lecht (1974) cites 
examples of findings from such projects: 

1. **Morc women of moderate ability enter occupational curricula 
whereas occupational men tend to be concentrated ai lower ability lev- 
els,*' according to a review pf several studies (p. 118). 

2. A survey of 50 junior colleges shows that'Vocational-technical col- 
lege students do- not differ greatly in self-reported high school grades 
Tom junior college students in general, but females tend to be superior to 
males" (p., 119). - 

3. A sample of students enrolled in Washington State high schools, 
who .plan to attend post-secondary business or vocational schools, re- 
ported lower grades, less interest in school work, and more dissatisfac- 
tion v/ith school than a sample planning to,attend college (p. 131). 

4. In the same sample, it was found that students' educational aspira- 
tions were influenced by their parents' educational attainments (p. 131). ' 

5. A survey of freshmen at a community college in Michigan shows 
that those enrolled in an academic curriculum evaluated their career 
potentials higher than did students enrolled in an occupational curricu- 
lum (pp. 140-141). . . 



TEACHER EDUCATION 

The goal of teacher education, as described by Evans and Terry (1971), 
is. to devise ways in which teachers can be prepared to teach accurately, 
effectively, and broadly so that their students will have maximum oppor- 
tunities to control their own future environments. Research in teacher 
education has not had a high priority in the vocational education r&d 
program funded under the 1963 and 1968 legislation. Consequently, as 
observed by Hamilton (1973), many researchers have concluded that 
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little is known about achieving teacher effectiveness or al^out the rela- 
tionship between teacher behavior and student growth. There have been 
some more recent developments in teacher education, but it cannot be 
demonstrated that they have been incorporated into existing vocational 
teacher education. Kievit (1975) reports that teacher education programs 
are most often determined by tradition or personal experience. She states 
that **the extent to which r&d has had an impact on teacher behavior 
through pre-service and in-service education is still largely speculative" 
(p. 32). 

Another reviewer of research in teacher education, Douglas Sjogren 
(1971a), notes that evaluations of teacher education programs are rare. 
Many are either accreditation ^eviews by regional agencies or by the 
National Council for Accreditation -of Teacher Education. Others are 
one-time-only evaluations of specific projects that-have little or no im- 
pact on future projects and have little utility for making decisions about ' 
programs while they are operating; this type of evaluation is appropriate 
only when limited resources preclude more extensive studyyaccording to 
Sjogren. Sjogren states that there have been no rigorous, objective, em- 
pirrcal, process-oriented evaluations. However, Schill and Allen (1974) 
did a follow-up study of 692 full-time teachers who had completed their 
education during the preceding five-year period; the sti^dy indicates that 
teachers in the teacher education program in California had learned to 
perform most of the tasks encountered in instruction. 

Another review of vocational teacher education (Swanson 1974) stress- 
es the inadequacy of the data base for reviewing and improving the 
preparation of teachers. First, there are few data on programs not spon- 
sored by the states, such as on-the-job training in business and industry. 
Second, data* on students are usually only enrollment data; data on insti- 
tution, program, occupational area, and duration of enrollment are not 
usually collected. Therefore, it is hard to assess even current demand for 
vocational teachers. Third, there are insufficient data on the current and 
future supply of teachers. Fourth, not enough is known about vocational 
teacher programs, for example, who conducts them, how the programs 
compare with one another, and how good their graduating teachers are. 

Despite limitations in knowledge available dn teacher education, some 
progress has been made by R&D in' finding more effective ways to alter 
teacher behavior. The identification of teacher competencies and re- 
search on pre-service and in-.scrvice education have been major subjects 
of exploration. An Ohio State research synthesis'monograph by Peterson 
(1973) reports several studies that determined the competencies required 
of vocational teachers. For example, Peterson cites three studies in agri- 
culture by Nattress, Kruskap, and Mitschele that examined competen- 
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cies needed by vocational agriculture teachers in crops and soil science, 
farm management, am^ animal science. 

Peterson (1973) also reports on an evaluation of xompetency-based 
teacher education (cbte) programs in Nebraska and Minnesota that 
show evidence of improved beginning teacher performance and highly 
improved student and teacher satisfaction. Using the cbte approach, 
teacher competencies are specified and prdspective teachers are held ac- 
countable for acquiring them. Hamilton (1973, p. 5) states that cbte is 
**primarily an outgrowth of the accountability movement in education," 
which is viewed as a radical innovation in education, but is not really 
different from earlier efforts. Hamilton states that cbte is based upon an 
inadequate research foundation (p. 21). She cites a report by Heath and 
Nielsen based on a review of 42 studies of cbte concluding that an em- 
pirical basis for cbte simply does not exist (p. 20). Hamilton cautions, 
that CBTE is "being oversold by usoe and creating unrealistic expecta- 
tions for widespread educational reform that could prematurely destroy 
the movement's potential" (p. vii). However^ advocates "Of cbte believe 
that teacher education will be emphasized in the future and that cbte 
programs will be developed on the basis of empirical knowledge of learn- 
ing. 

A considerable number of research projects haye been conducted in 
pre-service education, which is directed toward teaching novices the nec- 
essary competencies for effective teaching. Moss (1971) reviewed re- 
search designed to test some of the assumptions underlying pre-service 
programs and found that the best predictor of teacher effectiveness ap- 
pears to be academic achievement in teacher education programs. Moss - 
cited several studies indicating that- the number of years of teachers' 
experience in the occupation being taught is not correlated with student 
gains in verbal and manual skills (p. 45), Teachers' technical skills are 
important to student learning, but years of occupational experience do 
not necessarily ensure high levels of technical skills. Moss states that 
acquisition of technical skills by^ teachers via work experience usually 
takes longer than necessary; skills are too highly specialized to be maxi- 
mally useful; and by the time workers become teachers, the probability 
of pursuing further education has been reduced and their "worker value 
orientation may be too rigid" (p. 47). Also, a study by Cappiello reported 
by Moss indicates that student teaching has some benefits but that "it is 
frequently too little and too late" (p. 59). 

Research funds for pre-service and in-service teacher education have 
supported numerous workshops, conferences, and institutes as means of 
disseminating information to teachers (Kievit 1975). Relatively few stud- 
ies have assessed the influence of workshops and institutes on teacher 
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behavior, but there are some. Miller studied a ten-week summer appren- 
ticeship program for prospective teachers and found that it changed atti- 
tudes that improved teacher preparation (Peterson 1973). Techniques for 
micro-teaching, which deals with the division of instructional material 
into small or micro units, have been developed and evaluated for pre- 
service teacher educaticjn. Peterson cites one study by Bell that found 
* micro-teaching superior to traditional forms of teacher preparaiion. 
In-service teacher education is directed toward continuing the im- 
provement and development of experienced teachers. Research on in- 
service teacher education has compared the merits of various education- 
al feedback techniques forjmproving teacher skill performance (Peterson 
1973). Peterson reports that the use of video-taping techniques in supply- 
ing both pre-service_ and in-service teachers with feedback relating to 
their teaching performance has attracted much attention. Hoerneret al. 
conclude that the use of video-feedback was a benefice il technique in 
pre-service trade and industrial education workshops (Peterson 1973). 
ICelley et al. (I97I), in a study of the feasibility of remote supervision of 
home economics student teachers, found that teachers and supervising 
teachers expressed greater satisfaction with face-to-face p.nd video-phone 
techniques as opposed to audio-phone techniques. Hamngton and Doty 
(197 1) found that video-feedback of micro-teachiny techniques was 
effective and feasible for improving selected teaching skills of technical 
teachers.' 

It appears that r&d in vocational teacher education has had little ar»^- 
al influence to date upon improving teacher effectiveness, although some 
gains have been reported Few, if 'any, national priorities or state plans 
have included research and development' in vocational teacher educa- 
tion. 

INSTRUCTIONAL THCHNIQUES' 

Although a recent, comprehensive summary or synthesis of research on 
instructional methods in vocational education is not available, several 
instructional techniques are reported in the Ohio State review and syn- 
thesis monographs on vocational education content, industrial arts, com- 
munity colleges, adult education, education of th;; rural disadvantaged, 
and individualized instruction. ' . . 

Many instructional methods described in the reviews attempt to give 
.vocational instruction or guidance with substantially reduced interaction 
between student and teacher or counselor. Such methods include pro- 
grammed instruction (usually printed material), television, computer-as- 
sisted^guidance, teaching machines, multimedia packages, and a variety 
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of methods of self-instruction. These methods often are very costly to 
develop, requiring .specialized professional talent and. training to design 
and implement as well as much field testing before they can be u.sed with 
confidence. None of the metht)ds is unique to vocationaj education and 
only a small fraction of the research on those methods has been conduct- 
ed using vocational education content, students, or , settings. Vocational 
education research on these methods has used defective and Nveak meth-. 
odolog). As a result, many conclusions are tentative and only limited 
guidance can be given for the designation of methods for specific learn- 
ing situations. 

A second group of instructional methods in vocational education i.s 
characterized by settings for learning outside the classroom. Such set- 
tings are considered important in relating occupational preparation to 
learning appropriate interpersonal behavior and attitudes (Law 1971). 
Included in this category are cooperative education methods, clijiical 
practica, on-the-job training, vocational exploration, and a variety of 
special-purpose techniques such as those devised to assist the rural disad- 
vantaged: family-centered educational planning, pre-schbol preparation, 
vocational exploration and mobile training facilities. Much of the re- 
search on these extra-classroom instructional methods has been descrip- 
tive, developmeii'tal, or comparative. There has been only limited use of 
experimentation, and research methodology has often b.een defective. 
Like the previous group of methods, these methpds seem to affect learn- 
ing, but on the basis of limited evidence it is difficult to demonstrate their 
superiority over other methods or prescribe them differentially for partic- 
ular learning problem.s. 

Cooperative education programs train students by combiriing school- 
work with job experience. The planning and supervision of these pro- 
grams is done by both the school and the employers so that each contrib- 
^ utes to the students' education and to their employabiHty. An assessment 
of research in cooperative vocational education methods (Wallace L970) 
includes observations that apply to all extr^-xlassroom 'instructional 
methods. Wallace notes (1) the lack of a theoretical framework to guide 
the research and the applications of research findings; (2) the need for \ 
improved research technology for e\^aluation of such complex instruc- 
tion; (3) the heed for attention to many student-and teacher variables 
and their interactions in complex learning .environments; and (4) the 
need for attention to changes in students related to higher-level intellec- 
tual skills, social skills, attitudes, and self-identity. 

However, a recent study (Walsh and Breglio 1976) in large cities indi- 
cates that at the post-secondary level, cooperative education markedly 
increases tjie entry-level earnings o/ minority group members.. Moreover, 
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jobs for students in training were readily available except in one city, in 
which the unemployment rate was nearly 15 percent. 

The effectiveness of job training in industry- has been explored, and 
some of the benefits and drawbacks of job training at the work site have 
been identified. Little research has. been done on this beyond the docu- 
mentation of trends in the location of training (Wenig and Wolansky 
.1972). Very little is known about which tasks are taught most cfi'ectively 
on the job or in school. 

A third group of ins^tructional methods in vocational education, used 
in enrichment programs in schools for occupational learning, incluc^.c the ' 
simulations of occupational experience, te*im teaching, and individual- 
ized instruction. The job sunulation methods ofi'er the means of acquir- 
ing and practicing job skills in a controlled situation having instructional 
intervention, pacing for effective learning, minimum real penalties for 
error, and, substantially reduced training costs compared.with on-the-job 
learning. The efi'ectiveness of simulation seems critically dependent upon 
the aspects of the perforijiance setting being simulated, and the fidelity 
with which those aspects are smiulated (Fitzpatrick and Morrison 1971). 
Although most of the research on simulation has been done outside vo- 
cational education, notably in military and business settings, the evi- 
dence (for example, McClelland 1970) seems to be that simulation can 
be etTcctivc.in vocational instruction. 

Individualized instruction allows each student to progress in a pro- 
gram at his or her own pace. Reuarch on individualized instruction in 
vocational and technical education has demonstrated that'some individ- 
ualized instructional techniques have been efTective for specific student 
groups under specific conditions. However, results are typically neither 
conclusive nor gefteralizable. Esbensen's report (Impellitteri and Finch 
1971) of experiences in Duluth, Minnesota., states, **It is difficuh to state 
with assurance that individualized instruction is indisputably superior to 
traditional forms of schooling'' (p.:21). 

Although much of the research concerned with individualized instruc- 
tion is inconclusive, many vocational education programs are using tuto- 
rial laboratories to individualize instruction. For example, in Charlotte, 
North Carolina,' an occupational mix progr^.n in ten high schools sup-"- 
plements the regular vocational courses in business education. Teachers 
proficient in individualized instruction manage laboratories containing 
autotutorial materials. Regional occupational centers and programs in 
the wester: 1 states have also used individualized instructional laborato- 
ries to assist students with learning difficulties to attain the learning 
achievement necessary for entering the regular vocational education 
programs. ' 
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One kind gf instructional approach that is difficult to classify is the' 
residential school that often combines aspects of all three types of in- 
struction identified above. Warmbrod (1970) describes a dozen residen- 
tial programs as well as some Job Corps programs. Relatively little re- 
search has been conducted on these programs, but there is some favor- 
able evidence concerning graduations, job placements, and other program 
objectives. Most residential programs did not survive beyond the initial 
short-term federal funding period, 

A special case is the Mountain-Plains project at Glasgow Air Force 
Base, Montana, which has been supported by federal funds since fiscal 
1971, The Mountain-Plains project is described l^y the pi-ogram's di- 
rector, Bruce C Perryman (1975) as a family -oriented, residential pro- 
gram designed to economically rehabilitate disadvantaged rural families 
from a six-state region. Programs and services provided for participating 
families include career guidance; a career development program encom- 
passing the development of work attitudes and occupational prepara- 
' tion; limited medical, dental, and optical services; a family core curricu- 
lum that teaches home management, health, consumer education, parent 
responsibility, and recreation skills; and community and job placement 
services. Over 900 families have entered the program, including 22.6 per- 
cent representing minori^ groups. Project analysts predict the program 
will be cQst-beneficial to society, but some researchers question large 
expenditures for residential projects when r&d funds are limited. 

CURRICULUM DEVELOPMENT 

Curriculum development, demonstration, and research have been listed 
frequently among federal priorities for vocatiemal. education r&d since 
the passage of the Vocational Education Act of 1963. The federal* re- 
sources made available for curriculum work, augmented by substantial ^ 
investments by state and local agencies, have resulted in a large number ' 
of highly diverse curriculum projects. 

The development of xiew curricula does not always result in usable 
curriculum materials. When contractors or grant recipients have little or 
no functional relationship to the agencies involved- in instruction or 
teacher preparation, they may develop inappropriiate curricula. The test- 
ing of curriculum materials has sometimes occurred under highly con- 
trived conditions involving special incentives or preferential treatment 
for the testing sites but not for the operating programs.=Evaluations have 
rarely assessed the capacity of the cuiricula to continue as part of a 
nonexperimental instructional program. 

Much of the' curriculum development in vocational education has 
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been directed toward a small proportioh-of students and instructors in- 
volved ifunew occupations because they are said to need special program 
assistance. However,- the result is that the programs^ with large enroll- 
nients-— for example, home economics, business and office education, 
agriculture, and auto mechanics— are not supported by adequate curric- 
ulum development. . ^ 

Curriculum Content '..^ ' 

Several studies of vocational students and graduates in the 1960s indicate 
little' relationship between the pattern of program enrollments and the 
distribution of job opportunities in the community and found that rela- 
tively small proportions of vocational graduates obtained jobs directly 
related to their training. However, since those studies were completed, 
some progress has been made toward improving both the methods for 
selecting curriculum content and the relevanca-of curricula, for real job 
opportunities. ' 

Studies to .determine the content of curricula to be developed have 
used both primary and second^iry sources. Primary sources noted by 
Lar^n (1969) include mail surveys and interviews with teachers, em- 
ployers, and students; secondary sources include manuals, articles, and 
textbooks. In addition, Phipps and Evans (1968) note that subject matter 
^ is often determined by the opinions of experts. They state that develop- 
ment **is questionable when no validation studies are conducted to 
confirm the wisdom and biases of the 'experts' " (p. 372). The use of 
expert opinion in the development of curricula is noted by Carpenter 
and Rodgers (1970) as one of the primary sources for the development of 
agriculture education curricula. They state (p. 24): 

In practice, the typical curriculum builder in agricultural^ education considers 
result.*^ of a competency study of his own or of another investigator, seeks out 
expert opinion by formal' or informal surveys, and uses experienced teachers to 
assist in developing a curriculum outline and relaved teaching materials -. . . - 

vl f 

Some studies relying heavily on primary sources to determine the con- 
tent of curricula are reported in the Ohio State review and syntl;iesis 
series. Crawford (Ashmum and Larson 1970) interviewed workers and 
supervisors to determine critical tasks to* be learned by students in dis- 
tributive (sales occupations) education programs. Crawford's work 
served as a basis for the development of courses of study and materials 
for individual use by students. For office occupations, Bangs and Hille- 
stad analyzed interview respqnses from data processing managers, em- 
ployers, teachers, and executives and developed data processing curricu- 
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la and^ course outlines (Price'and Hopkins. 1970). Similarly, Carter sur- 
veyed several hundred local office supervisors to identify which tasks 
should receive more or less .emphasfis in.^business occupations curricula 
(Price-and Hopkins 1970), ^ r • . 

Another approach to improving^fhe correspondence between vl)ca- 
tional^urricula and jobs has been the development of task analysis. Task 
analy^is.j^a method of examining the various tasks .required in a,job and 
identifyin^their characteristics and the skills required for them. Methods 
for constructing and analyzing job task inventories,, developed by the 
,<iU,S. 4ar. Force (ChristaM974),- have bden' adapted for Application to 
'civilian occupations (for. example, Melchingand Borcher 1973)', Innova- 
tive methods have been, devised for identifying theYechnical concepts 
required-fpr-eftectiyc7perfdrmance of various occupations (Mos.s et al. 
1970). Com.pitter 'programs have been developed to perform the. data 
processing required by the new methods (for example. Mead 1972, Sta-^ 
cey 1974), ^. ■ . 

In order to. improve the' relevance of cu rricula to the job market, cur- 
ricula have been developed for new occupat-ions, such as biomedical* 
equipment technician, electrb-mechanical technician, laser and electro- 
optical technician, a variety of allied health occupations, and computer 
science occupations (Simpson 1975), Traditional programs have also 
been revised to correspond more closely with current emf^loyment Op- 
portunities, For example, the nob£Ls project (Lanham et^al, 1972) was 
designed to develop and keep current a new secondary school curricu- 
lum in business and office occupations. Data were gathered on the per- 
formance requirements of newf oflice occupations (Huffman and Gust 
1970) as well as on current jobs; a comprehehsive set of educational 
objectives-was prepared for which efficient learning conditions could be 
devised. 

Student Needs . ^ • . ' 

Ten years ago, the vocational education programs in most schools were 
limited in .variety, and entrance standards were sometimes as severe as 
for academic programs. Recent reviews of vocational education curricula 
(Maxwell et al. 1 973,' Oakleif 1971, Locketteand Davenport 1971, Simp- 
son 1975) suggest that more curricula now offer a broader range of edu- 
cational and vocational choices and will accommodate the needs of- 
many more students. 

Phipps and Evans (1968) list curricula that have been developed for 
dropout-prone students and for disadvantaged rural youth, Morrison 
(1970) cites program's to prepare disadyanta^ed students for healthy food. 
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education, and service occupations. Simpson (1975), reporting a trend 
toward paying more attention lo students' characteristics and needs, 
cites among other curricula a source book developed to assist in devising 
programs for inmates of correctional institutions, career education mate- 
rials for Spanish-speaking children, and business management curricula 
for minorities and disadvantaged learners. Stutz and Merrell (1967) de- 
scribe the development of.a vocational program designed for students in. 
"small, isolated, rural schools. The Center for. Vocational Education 
(1975) developed curricula for deaf students and continues to prepare 
career education materials for them. Curricula also hav.e been completed 
to prepare Indiari students to be advanced electro-mechanical techni- 
cians (Terry et al. 19'/5). 

In another, attempt to meet students' varying needs, individualized/" 
curricula, have been developed to enable any stlident to enter a program 
at a level determined by his or her own capabilities. Each student can[ 
then acquii*e vocational training in an individuaLse^uence that provides \ 
preparation for occupations at successively higher levels. SQch a program \ 
can accept virtqally all students and provide each with vocational: comJ 
petence at whatever level is permitted by the student'? interests, abilities, 
^ motivatio;!, and time. . i^; 

. Several programs have been undertaken to implement mdiyidualized 
curricula. The.{5re-engineering technology program, or Richmond Plan 
(Asbell. 1967), employed the basic concept but limited entrance to stu- 
dents of at least average ability who'were underachieving. Project feast ' 
(Batmale* 1966), which adapted.the Richmond Plan to food, education, 
and service "technology, accommodated students at all levels of ability 
and piovided a graduated sequence of occupational goals. An attempt 
was made to use this strategy comprehensively in Project able of the 
American Institutes for Research and the Quincy, Massachusetts, Public 
Schools (1964): individualized curricula were developed in 1 1 broad vo- 
cational areas involving over 200 occupations organised into more than 
30 sequences. 

Occupational Adaptability 

Because individuals can expect to shift occupations several times during 
their working lives, it is important to design vocational curricula that 
provide a useful basis for occupational versatility. Phipps and Evans ^ 
(1968). identify and illustrate several strategies that have been used to 
determine such curriculum content. The transferability approach at- 
tempts :o identify content with high transfer value .(generalizability) for 
many types and levels of jobs. The competency pattern -approach at- 
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tempts to identify patterns of competencies or-skills needed in-occupa- 
tional areas as contrasted with studies of single job' titles. The functions 
'^of industry approach analyzes those functions from wliiCh the education- 
al needs of workers are deduced. 

By far, the most commonly, used strategy has, been the cluster ap- 
proach, in which learners are prepared for a group or cluster of occupa- 
tions. The strategy identifies requirements common to ;seVeral current 
occupations and includes preparation fcfr these shared require^ments .in 
the curriculum (SjogreQ^and Sahl 1966).. The cluster approach has been 
used to develop curricula for technician^ (Schill and^Arnold 1965), office 
workers (Perkins et al. 1968), bu;lding trades workers (Bakamis et al. 
1966), construction workers (Franlz 1967), and agricultural and' metal 
workers (Sjogren et al. 1967)'. Simpson (1975) reports the development of 
curricula in each of^he 15 occupational clusters, identified by usoe. A 
clustering system, having curriculum jmplica'tions. for the comprehensive 
oareer education model is reported by Taylor et -al. (1972). 

Significa/it methodological and theoretical problems are faced by:re-i 
searchers at^mpting to identify commoRYequirements in existingjobs or 
to provide 'prejDaration for a yariety of job opportunities. Sjogren (1971) 
notes th^it jobs have most commonly been clustered by lisirlg an arbi- 
trary definition of similarity; this" method, altHqugh subjective, is- none- 
theless quite reliable .'(repeatable). Sjogren reports ^e, frequent use of 
statistical teol\riiques such as factor analysis' to define clusters by analyz- 
ing ratings or other quantified judgments of job or task characteristics. 
Sjogren (I960) notes that no matter what -kinds of measurement and 
analysis techniques are used, clusters are determined in large part before 
/the analysis, when certain jobs are selected fpr study and others are not. , 
Any job or set of tasks would probably sort into different clusters -de- 
pendfng uppn the mix of jobs studied. Further, since any job is an arbi- 
trary coHection of 4asks that varies from instance to instance, each job 
would probably" sort differently- depending on the pa/ticular job sites 
studied. Clusters have not often been studied across skill levels within 
job hierarchies, although Such studies would identify curriculum content 
^ important for career progression. ^ 

Curriculum Integration ^ • 

Gj^gne (1965) states that the goals of education, include satisfaction with, 
,wdrk or vocation, responsible citizenship, and participation in a variety 
of aesthetic experiences. Vocational eclucators have recognized the need-' 
to prepare students for more than just the job skills required for a partic- 
ular occupation, and they have begun to emphasize the need to integrate ^ 
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vocational and other curriculum elements. Attempts to provide more 
comprehensive, integrated programs have not usually included the exten- 
sive analysis and development that seems required for a new curriculum. 

Three projects illustrate such atrenipts to produce comprehensive inte- 
grated curricula. Project able (American Institutes for Research and 
Q.uincy Public Schools 1964) attempted to develop a full secondary 
school curriculum to accommodate all students who were not preparing 
to enter a four-year college program. The project strove to prepare stu- 
dents for vocational competence, responsible citizenship, and self-fulfill- 
ment (Morrison and Lecznar 1966). Numerous difficulties were encoun- 
tered (Morrison 1968) and the resulting curriculum was neither as com- 
plete nor as well integrated as was intended. 

The second project," called Educational System for the 70's or es '70 
(Bushnell 1967, Bushnell and Rubel 1968, E F. Shelley & Co. 1968), was 
a larger and more ambitious undertaking by an organization of 17 school 
systems across the country in cooperation with the U.S. Office of Educa- 
tion. The goal was to develop a secondary school curriculum from which 
graduates could choose t£Lenter a four-year college, a junior or commu- 
nity college, advanced vocational programs, or gainful employment. Ob- 
jectives for the curriculum, stated as performance skills to be acquired by 
students, were to be assembled from a variety of sources, including voca- 
tional and academic education, and consolidated" to define the major 
structure of the trial curriculum. This large, expensive project was dis- 
continued (during an admmistrative reorganization of uSOE) before work 
on a substantial portion of its objectives could be initiated. 

Finally, the career education model programs (Goldhammer and Tay- 
lor 1972) were attempts to use various settings (home, school, communi- 
ty, work place, speciaK residence) to integrate and give wider meaning to 
the elements of an educational program. Different integration strategies 
were used: infusing career awareness, career exploration, or career prep- 
aration into existing curricula; and using a life situation (such as a work 
task) as the vehicle for learning related skills (such as calculation). The 
models varied in their structure and methods, but each attempted to 
develop adaptable systems for the integrated learning of important skills. 
None of these models was permitted to develop as originally planned, 
and all terminated short of their original objectives (after an administra- 
tive reorganization in hew). Even though some of the products and re- 
sults have been disseminated, no integrated career education curricula 
are ready to be installed in schools. 

None of these massive programs has successfully constructed an inte- 
grated curriculum that has been adopted by others. The failure may be 
due to the financial Costs of such development, to the size and complexi- 
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ty of the task, to the time required for development- or to practitioners' 
suspicions of centrally developec curricula. However, curriculum inte- 
gration is still viewed as a desirable goal. Medium-sized curricula that 
have been tested on students and revised, and for which teachers have 
been trained, have been adopted widely. 

Curriculum Evaluation 

According to Larson (1969), designirig curricula involves the setting of 
training objectives. Pilot tests and evaluations: to determine wh-ther ob- 
jectives are being met have been incorporated into several large-scale 
curriculum projects. Householder (1972) describes the American Indus- 
try Project, which developed an instructional program to help students 
understand basic concepts of industry and which was extensively held 
tested, evaluated, and revised. Evaluation studies found that students 
enrolled in this program had more positive attitudes towards its courses 
than towards others, acquired knowledge of job opportunities in indus- 
try, and showed greater interest in seeking industrial employment than 
they had shown prior to exposure to the program. 

The Industrial Arts Curriculum Project, also described by Household- • 
er (1972), included components of field testing and in-service teacher 
education as well as curriculum development. Two courses were devel- 
oped by the project— the Worid of Construction and Worid of Manufac- 
turing. Developed materials were field tested throughout the country 
over a four-year period, evaluated, and revised. Subsequently, the project 
"attained a new milestone" (p. 20) by making commercially available ah 
instructional system for industrial arts. 

Householder also cited an evaluation of a project that provided indus- 
trial education experiences at the secondary, community college, and 
university level and emphasized the interrelationships between industry 
and other social institutions. The evaluation showed that students en- 
rolled in courses in the project improved their attitudes toward school 
and had better attendance after participation. 

Despite the positive results of the evaluative research noted in industri- 
al arts education. Householder notes that "the body of knowledge upon 
which industrial arts courses is based has not yet been fully defined, 
categorized, and communicated" (p. 43). Similar conclusions have been 
drawn by reviewers of research studies in health occupations, technical 
education, agricultural education, and home economics education (Hoi- 
loway and Kerr 1969, Phillips and Briggs, 1969, Carpenter and Rogers 
1970, and Nelson 1970). Nevertheless, in the Committee's hearings and 
in interviews, respondents mentioned curriculum "projects more consist- 
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ently than any other type of vocational education r&d as having had 
, impact on practitioners. 

LABOR MARKET SUPPLY AND DEMAND INFOR^lATION 

Projected labor market supply and demand information in specific occu- 
pations, particularly at the state and regional levels, can be quite useful 
in determining the types and amounts of vocational and technical educa- 
tion required for , the future. Kelley et al. (1975) iderilified more than 300 
projection studies over the period 1965-73 that have been based on local, 
state, and national data'. The two general types of demand forecasting 
techniques are employer surveys and analytic projections, such as trend 
extrapolation. Kelley .et al. state that in the studies they reviewed, 51.4 
percent used employer surveys and 48.6 percent used analytic projec- 
tions (p. 128). The m?]oT problem associated with both methods is that 
they cannot foresee ali the elements that would cause trends to change, 
create demands in new occupations, or cause old occupations to become 
obsolete. 

Two techniques of labor market supply analysis— multitrack and sin- 
gle-track research — have been discussed by Young et al. (1972). Multi- 
track research attempts to evaluate alternative training methods or devel- 
op data systems for evaluation. For example, Foster studied sources of 
trained personnel in the construction industry and found that vocational 
education is not supplying a "significant" number of skilled workers 
relative to other types of training (Young et al. 1972). He also reports 
that oa-the-job training was judged by workers to be better than class- 
room training in helping people acquire skills. Similarly, Horowitz and 
Hermstadt analyzed workers in the tool- and die-making trade and 
found that many accomplished craftsmen were high-school dropouts 
(Young et al. 1972, p. 38). However, they also found that vocational high 
school combined with apprenticeship seemed to be highly effective in 
training workers. 

Single-track research analyzes a particular type of vocational training, 
such as apprenticeship, on-the-job training, or military service training. 
Young et al. (1972) report that a national survey shows that construction, 
machinist, and tool- and die-skilled workers believe apprenticeship is a 
helpful way to learn their trades. Several studies indicate that on-the-job 
training is highly valued by both industry and employees (Young et al.). 

Studies of labor mobility also help to determine future labor supply. 
These studies report fairly consistently that-a high proportion of second- 
ary school graduates find and keep jobs in or near the community in 
which they have attended school. For example, Eninger reports that ap- 
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proximately 80 percent of a national sample of 1953, 1958, and 1962 
trade and industrial graduates had not moved to another city for em- 
ployment purposes (Young et al. 1972). Labor market forecasts are most 
frequently done at state and local levels. Kelley et al. (1975) found that 
12. 1 percefnt of the studies are national in scope, 46.3 percent are state 
forecasts, and 41.6 percent are local. They also report that national fore- 
casts across all occupations are recent, limited, concerned more with 
demand than supply, and dominated by the projections program of the 
Bureau of Labor Statistics. Most other national forecasts reported are 
related to specific occupations. 

Kelley et al. conclude thei: review of labor market forecasting by stat- 
ing that there are serious limitations in the studies they reviewed. In 
addition, Kaufman and Brown note that it is almost impossible to pre- 
dict future supply or demand because of the many external influences on 
the labor market; many projections have proven to be inaccurate (Price 
and Hopkins 1970). 

The Ohio State monograph series reviewing. occupational research re- 
ports labor market research in nearly every ^rea of vocational training. 
In agricultural education, for example. Heady and others have made 
estimates of labor requirements in farming occupations to 1980 (Carpen- 
ter and Rodgers 1970). Studies of farming opportunities, which are pri- 
marily local in scope, are appropriate because farmers tend to live within 
25 miles of the schools they attended. General conclusions drawn from 
these studies a/e that aggregate farm employment, perhaps much of it of 
a marginal type on less productive land, has been declining, as has mar- 
ginal employment in other sectors of the economy. On the other hand, 
the work force in nonfarm agricultural occup:.tions is increasing. Many 
studies conclude that there is also an inadequate supply of off-farm agri- 
cultural workers (Carpenter and Rodgers 1970). 

Price and Hopkins (1970) report that .studies indicate that the occupa- 
tional opportunities for business education graduates will continue to 
grow. Like the research on farming opportunities, local research is most 
useful to vocational educators since business graduates tend to work 
close to the schools from which they graduated. 

Ohio State reviewers also report many local studies in home econom- 
ics, health, trade and industrial, sales, and technical occupations. In gen- 
eral, growth is expected in sales occupations, public services, human 
services, such as social work, and environmental occupations, such as 
sanitation. 

Studies of labor market demand for vocational education personnel 
are of special concern to vocational education. In 1971, Somers reported 
uncertainties in projections and disagreements about the likelihood of a 
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shortage of vocational education teachers in 1975. In the past, forecasters 
failed to estimate correctly the flexibility and adaptability of the sources 
supplying instructional personnel. Although icar.onably accurate locr^i 
projections have been made, Somers is critical of the few national projrc-^ 
tions of teacher supply and demand. He blames the lack of data and 
inaccurate projection techniques for the fact that various national studies 
have produced varying estimates of the size of predicted teacher short- 
ages. ' ^■ 

Both the quantity and quality of labor market supply data in vocation- 
al education have improved in the last few years. Many people are trying 
to improve demand information, which is much more difficult to predict 
accurately than supply information. Several states have developed and 
instituted management information systems containing labor market 
supply and demand data for use in planning vocational education pro- 
grams, r/*^ 

ADMINISTRATION OF VOCATIONAL HOUC ATION 

Studies of the administration of vocational education have been re- 
viewed by Ralph C. Wenrich (1970) for the Ohio Center for Vocational 
Education. In federal, state, and local administration, topics of interest 
include policy making, administrative structures, and program and facili- 
ty, planning, financing, and staffing. Studies of federal policy making 
have identified changes in past policies and recommended improvements 
based on the judgments of the researchers. The role of state departments 
of education in vocational education policy has also been studied. For 
example, Swanson studied stale-level vocational education administra- 
tion and found that "the primary emphasi.s was ... on compliance 
(checking and regulating) and the secondary consideration was change 
and leadership - (Wenrich 1970). Other studies in state policy making, 
such as Frigiola's study in New Jersey, have provided the basis for rec- 
ommendations for improving a state's vocational education services and 
programs (Wenrich 1970). 

There have been Yew studies of the federal administration of the voca- 
tional education R&D program. Some'state*level studies have attempted 
to describe the structure and function of area vocational schools or other 
state vocational programs, but few have analyzed the role of research 
coordinating units or state departments of education with respect to R&D 
or vocational education programs. 

Studies of program planning at the state and local levels are usually 
preceded by studies of labor market demands. Researchers often attempt 
to match existing programs with projected labor market needs. For ex- 
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ample, Hendrix designed a data collection instrument to determine the 
labor market needs of the community, the goals of the people, space and 
equipment availability, and special instnictional needs (Wenrich 1970). 
Recent understaffing at vocational schools has been a problem, and 
some researchers have attempted to identify new sources of teachers. In 
Michigan, Messerschmidt studied recruitment and hiring-^practices and 
determined that the "primary source of part-time instructors was local 
business and industry, and, attempts to use retired industrial and inilitary 
personnel were not successful" (Wenrich 1970, p. 35). Wenrich notes that 
a high proportion of the projects on administration are surveys involving 
the use of mail questionnaires. AUempts to generate information about 
administration are improving, but generally lack sophistication. He 
states (p. 56): ' ' 

We need controlled experiments, some of which would, of necessity, extend over 
a period of years and would involve data based upon observation rather than 
mere opinion. 

Many researchers, developers, administrators^ and practitioners cite 
the devel6pjnent of management information systems (miss) as one of 
the most significant accomplishments of the past ten years of vocational 
education r&d. John Evans defined educational miss as systems that 
"convert data into information of use to managers* at different levels, 
places and times in the decision making process" (Hale 1971, p. 69). mjss 
have been developed by several state, regional, and even national organi- 
zations. For example, the '^System for State Evaluation of Vocational 
Education'' (Hale 1971) developed by the Center for Vocational Educa- v 
tion contains data on pupil characteristics, program characteristics, and 
employment rates. Management reports produced by the mis can show 
information such as the ethnic distribution of students by programs. 
Status reports on students' can also be produced,.enabling researchers to 
periodically follow-yp students who graduate from or drop out of voca- 
tional education programs. — 

Generating state-level data about vocational education students and 
programs is a major task of the rcus. Some rcus have developed miss 
with federal R«feD funds. For example, the Tennessee rcu developed an 
MIS containing follow-up data and an occuational information system 
contrJ^ing descriptions of jobs and vocational or career programs for all 
gr?idt ievels. The Tennessee mis, developed for reporting purposes, is 
used to resi md to requests for data, analyses. This information, along 
with the Rcu's supply and demand projections, is used in program plan- 
ning. Researchers have developed state-level vocational guidance sys- 
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terns, containing data on job openings 'obtained through employer sur- 
veys, and designed to match young people with jobs. 

Regional management information sy.stems are often more efficient 
than state-level systems since costs and benefits can be shared by many 
school districts. Hale (197 1) describes the Midwestenr'State&^ducation- 
al Information Project, which developed a system for collecting data on 
facilities, financing, instructional programs, personnel, and students that 
are comparable among local school districts and among cooperating 
states. 

In addition to miss, other techniques have been developed to facilitate 
management; for example, trend analysis is used to forecast student en- 
rollment data. However, adequate data bases and relatively static condi- 
tions are necessary for trend analysis to be accurate. Hale (1971) states 
that often expert opinion is more valuable in planning vocational educa- 
tion -programs than trend analysis because experts have experience that 
enables them to deal with a number of factors that are not formally used 
in trend analysis. 

EVALUATION OF VOCATIONAL EDUCATION' PROGRAMS 

The literature describing the evaluation of vocational education pro- 
grams is discouraging; it yields little useful information for vocational 
educators. The research designs have.used analytic procedures requiring 
simple quantitative input and have failed to encompass many important 
educational issues. Evaluations have used research methods that are in- 
compatible with the complexity of the learning, teaching, and adminis- 
trative situations. 

Brief reviews of what are considered to be some of the best evaluations 
of vocational education in the last ten years are presented in thi.s section. 
The specif - criterion variables studied are vocational graduates' knowl- 
edge aboui occupations, their job readiness, job satisfaction, and earn- 
ings. 

Occupational Information 

The amount of occupational information possessed by people entering 
the labor force may be an important determinant of future success. 
Decker Associates (1967) found hat, compared with students in academ- 
ic or general .curricula, vocational graduates know slightly less about 
occupations and are more likely to name skilled trades as the best jobs 
known to them. Other students cite professional, technical, and manage- 
rial jobs as best. However, since Decker failed to control for socioeco- 
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nomic factors, the observed differences may be due to the differing back- 
grounds of the students rather than to the curricula in ^vhich they were 
enrolled. Using data on male high school graduates who did not attend 
college, Grasso (1972) found no evidence to indicate that students from 
vocational, general, and academic curricula differ in their knowledge 
about occupations. Since Grasso controlled for variables such as apti- 
tude, family background, grades, and work experience, his conclusions 
are probably more valid than Decker's. 

Although career education emphasizes knowledge about occupations, 
some vocational educators doubt that it is a valid measure of program 
success, since it may not be directly related to the quality or appropriate- 
ness of curricula. 



Job Readiness of Vocational Graduates 

Some researchers expect that a vocational graduate will be ready for the 
responsibilities of a particular job without further training, so that a 
program can be considered successful if its graduates infrequently desire 
immediate further training. ^ 
Grasso (1972) found that approximately 85 percent of all high school 
graduates who have not attended college express a desire for additional 
training. Among whites but not blacks, nonvocational graduates are 
more likely than vocational graduates to feel a lack of education or train- 
ing. It is not surprising that academic students most prefer to attend 
college. However, for both blacks and whites, there is almost no differ- 
ence in the desired type of additional training between the general and 
the vocational graduates. Thus, one might question the view that voca- 
tional education graduates regard themselve.s as better prepared for em- 
ployment. . 

Grasso found little variation among whites from different high school 
programs in the kind of training they seek (which included business col- 
lege, company school, apprenticeship, military s^ervice, and others). How- 
ever, the types- of post-secondary training actually received by the differ- 
ent graduates does vary; academic graduates report twice as much pro- 
fessional or technical training as graduates of the other programs, where- 
as commercial students report greater managerial training, and vocation- 
al students more skilled manual training. It appears that blacks partici- 
. pate in additional training to a lesser extent than whites. 

Kaufman and Lewis (1968) found that almost 90 percent of the voca- 
tional graduates said their programs had made a "real effort'' to prepare 
them for employment. This figilre appears less impressive when one con- 
siders that almost 60 percent of the academic students said the same 
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about their programs, which put less emphasis on preparing students for 
"immediate employment, ^ 

It must be noted that job readiness may have validity only as a short- 
term goal. All too often specific training for a job immediately after 
graduation becomes less valuable when jobs change or. a student's or 
worker's interests change. 

Job Satisfaction . . 

Kaufman et al. (1967) found no significant differences in job satisfaction 
reported by graduates of vocational, academic, and general high school 
programs in the Northeast. Eninger (1965, 1968) found that trade and 
industry graduates have greater job satisfaction than do those from nori- 
vocational curricula. Job satisfaction tends to be greater among those 
whose jobs relate to their training. 

Garbin et al. (1970) found no difference between, vocational and non- 
vocational programs in the degree to which their graduates report being 
hired at anticipated levels, achieving expected income, or "coping" with 
jobs. Grasso found that both black and white academic graduates and 
white commercial graduates hold more favorable career positions than 
do others, and that vocational graduates do not differ from general cur- 
riculum, graduates. White academic and commercial graduates are found 
to have higher overall job satisfaction than graduates of the general cur- 
riculum, while the latter group is not significantly different from voca- 
tional graduates. However, for blacks, job satisfaction, which is much 
lower than for whites, does not vary significantly with thei; curricula. 

Earnings 

Much of the research on the effects of vocational education programs 
compares the earnings of vocational education graduates with either gen- - 
eral or academic curricula graduates. Differences in starting pay and in 
the progression of pay with increased experience are often investigated.- 
However, since many of the studies do not control for variable., related 
to both the choice of high school curricula and measures of labor market 
success, their conclusions are suspect. 

In a national study of the graduating classes of 1953, 1958, and 1962, 
Eninger (1965, 1968) found different results at different points in time. 
For example, he reported that the extent to which training is related to a 
graduate's job seems to affect wage rates in some samples but not in 
others. Eninger *s conclusions are further confounded.because he failed 
to use'availabh control variables. 
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Persons et al. (1968) performed a cost-benefil study of a farm business 
management program conducted by public schools in Minnesota to im- 
prove both technical and entrepreneurial skills of farm operators. Criteri- 
on variables, which were the farm operator's labor earnings, returns to 
capital and family labor, and total farm sales, were adjusted for yearly 
fluctuations in annual farm income. Persons et al. found that benefits to 
both the individual participating farmers and the community outweighed 
direct and indirect costs. The study resulted in considerable growth of 
the farm business management program and in state legislation for pro- 
gram support. 

Two recent studies of vocational education graduates featured special 
attempts to assess the effects of education on future earnings. Stromsdor- 
fer (1972) analyzed the National Longitudinal Survey data for both 1966 
and 1968. All male, out-of-school youths-^with widely varying amounts 
of formal education-were treated as a single group. He chose to indi- 
cate annual income by multiplying earnings of the survey week by 52, 
thus reflecting differently from other researchers the influences of over- 
time, job change, multiple jobs, and time not worked. Stromsdorfer 
found no sigmficant differences across curricula in the 1966 data, but in 
1968 he found that former vocational students were earning about $400 
more per year than academic graduates and approximately $275 per year 
more than the general curriculum graduates. 

Grasso (1972) investigated the effects of curricula on the hourly rate of 
pay. For black males, he found no significant curricular. differences; even 
work experience and post-secondary school training did not help to ex- 
plain differing wage rates among blacks. For white males, curricular di- 
fferences were not found to be related to the starting wage rates, but 
white male vocational graduates apparently benefited more from addi- 
tional training than did graduates from other curricula. 

Follow- Up Studies 

At least two basic kinds of information to serve two different purpc^ 
can be collected in student follow-up studies: students' opinions about 
their programs can be used directly to alter the programs, and data on 
the students' performance (employment information) can be used to 
evaluate the success of the programs. Allen (1975) collected both types of 
data in his three-year follow-up study of 364 California vocational grad- 
uates. Allen overcame a common problem of follow-ups, locating. the 
students, by using addresses supplied by the Department of Motor Vehi- 
cles, thereby maintaining contact with 97 percent of the study group. At 
the end of the 1973-75 period. Alien found that 75 percent of the respon- 
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dents were working, and 72 percent had jobs related to thein high school 
vocational training. Seventy percent said the high school vocational 
training was helpful in their present jobs, and 88 percent would recom- 
mend vocational training to other students. Fifty percent had enrolled in. 
advanced training, and 49 percent had had additional training on the 
job. When respondents were asked about what changes they would re- 
commend in the program, 71 percent of the respondjents recommended 
more applied practice; 66 percent, more job-related information, and 67 
percent, better help in job placement. Most seemed satisfieci with their 
teachers. Allen concludes (p. 25): *There is no doubt that follow-up 
studies can provide schools with data and information necessary for in- 
structional modification and improvement." 
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Sources of 
Information 



The Committee used a wide variety of sources to evaluate vocational 
education r&d and to describe the structure and management of the r&d, 
program. In addition to the literature review reported in Appendix A, the 
Cdmmittee and its staff reviewed evaluations of vocational education, 
visited ten state-research coordinating units (rcus), held hearings, inter- 
viewed usoe personnel, and conducted a mail survey and a series of 
telephone interviews. It should be noted that comments made by person- 
nel about their own organizations were verified by others outside those 
organizations whenever ix)ssible. 

Two major projects evaluating the national r&d effort were reviewed, 
a^ well as listings and descriptions of single projects reported to be suc- 
cessful; Examples of successful projects were cited by usoE, the South- 
wide Research Coordinating Council, and Committee members. 

Papers covering a wide range of topics were commissioned by the 
Committee and prepared by IS prominent vocational educators and re- 
searchers. In addition, one Committee member prepared a paper on vo- 
cational education and women. A list of these papers is presented in 
Table Bl. 

^ - "■ 

Visits to state RCUs, which are responsible for the management of the 
vocational education r&d effort in each state, provided another source of 
information for assessing r&d and describing its administration. In order 
to gather relatively complete data on the Rcus—their functions and the ^ 
impact of R&D in their states— ten states were chosen for comprehensive' 
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TABLE B 1 Commissioned Papers 
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Author , " 

Edwin L. Herr, Pennsyhania State 

University 
Jacob J. Kaufman, Pennsylvania State 

University 

Merle E. Strong, University of Wisconsin 



Carl J. Schaefer, Rutgers University 
Grant Venn, Georgia State University 



Mary B. Kievit, Rutgers University 



Roman Pucinski, Chicago, Illinois 

Teh-Wei Hu, Pennsylvania State Univer- 
sity and Ernst W. Stromsdorfer, U.S. 
Department of Labor 

Phyllis D. Hamilton, Stanford Research 
Institute 

Garry P . Bice, RCU-Knoxville, 
Tennessee 'i , 

Joseph F. Blake, Millersville State. 
College 

f. - 

David S. Bushnell, Human Resources 

Research Organization 
Carl E. Thoresen, Stanford University 

and Craig K.^Ewart, Stanford 

University . - 
Henry M. Brickell, Policy Studies 

in Education 
Elizabeth J. Simpson, University of 

Wisconsin<Madison 



Title 

Guidance and Counseling, Vocational Edu- 
cation, Research and Development 

Human Resource Development and Voca- 
tional Education 

The Status of Re.scarch Capability in Voca- 
tional Education Research and Develop- 
ment 

Helter-Skelter: Vocational Education R&D 

An Analysis of Vocational Education Re- 
search and Development Policies from 
Three Perspectives 

Vocational Education Research an^ Devel- 
opment as a Factor Influencing Teachers, 
to Change Practices 

Vocational Research and Development: Key 
to Survival in America's Third Century 

An Analysis of the Impact of Applied Re- 
search and Demonstration Projects in 
Vocational Education 

Vocational Education Research and Devel- 
opment for Ethnic Minority Students 

An Analysis of Dissemination and Utiliza- 
tion of Vocational Education Research 
and Development Efforts 

Dissemination of Research and Development 
Products and Results in Vocational 
Education 

Policy Alternatives in the Evaluation of 
Vocational Education 

Careers, Counseling, and Control 



A Framework for Developing Alternative 
Scenarios for Vocational Education R&D 

Curriculum Development in Vocational- 
Technical Education: The Part I Program 



NOTE: Paper added to series by Committee member Pamela A. Roby, "Vocational 
Education and Women.'* 



Study, States were selected to be representative in four ways: by region, 
by amount of federal RCU allotment, by institutional location, ar}d by 
administrative responsibility relative to Part D. RCUs in two states were 
visited in each of five regions: Northeast, Southeast, North Central, 
South Central, and West. Four states receive relatively large^allotments 
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from the federal" government; five,. medium-sized aUotnfents; and one a 
smal allotment. Twoor the ten rcus are based primarily at.a univershy! 
the other e.ght are based ,n a division of the state depirtipent of educa- 
tion. In seven^states, the rcu directors administer sfate Part D funds- 
RCUS ,n the other three do not. The Committee.lso cliose RCUs^a had ' 

on L hi"" ]^ n "'^^"^ "'^'"^'^ ^'S^ quality programs- judged 
on the-b^is of.Committee and staff conversations with many rcu di- 
, rectors and USOE per?onnel. In addition to the directors of the ten rcus 
fn char ' ^^-'"- ^'f interviewed a few former rcu directors? those 
pcu staff admm.stration and dissemination, and additional 

fi Jl'.S'"""^ ^ ^?ries of hearings^in May 1975 to gather 

,S eH 'f°™^"on^from many people -knowledgeable about voca- 
uojjal education r&d. Twenty selected organizations were invited to par- 
ticipate m the heanngs. Two organizations-one representing the chief 
state school officers and the other representing proprietary's hools- 
chose not^to participate. Two others, representing organized student 
groups and vocational researchers, were unable to attend, although the 
esearchers did send written information. Iri addition to the orgfniza! 
ions, a small number of individuals representing college and university ^ 
eadership m vocational education were invited to participate. A o S 
24 people parlicipaied: ^ - . ^ y^^i kji 



STATE LI-ADERSHIF OKGANIZATIONS 

State Advisory Council on. Vocational Education 

John A. Beaumont, Chairman, State Advisory Council on Vocational " 

Education (Florida) 
William Bruce Howell, Executive Director, Statfe Advisory Council on ' 

Vocational Education (Flgrida) • ' ^ 

Council of Chief State School Officers 
(Chose not to participate) 

National Association of State Directors ojT Vocational Education 

Carl Lamar, Assistant Superintendent for Vocational Education (Kentucky) 

National Association of Research Coordinating. Unit Directors ''^ 

Ronald D. ^kCage, rcu Director (Illinois) 

Garry Bice, rcu Director (Tennessee) 
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LOCAL LEADERSHIP ORGANIZATIONS 

National Council of Local Administrators * 

Fred Miner, Assistant Superintendent (Lakewood Center, Washington) 

American A ssociation of School A(lmin istrato rs 

Abram Friedman, Assistant Superintendent, Division of Career/Continuing 
Education (Los Angeles, California) 

National Association of Large City Directors of Vodational Education 

B. J. Starnps, Assistant Superintendent fcr Career Education (Dallas, Texas) 

National Coordinating Council for Vocational Student Organizations 

Mildred Reel, National Advisor— Future Homemakers of America 
(Washington, D.C.) »--0> 

Proprietary Schools 
(Chose not to participate) 

COLLEGE AND UNIVERSITY LEADERSHIP 

Junior Colleges 

Dwight Davis, Dean, Joliet Junior College (Illinois) 
Universities 

Keith Goldhammer, College of Education, Michigan State University 
Alfred H. Krebs, Acting Vice President for Academic Affairs, Virginia 
Polytechnic Institute 

OTHER 

American Personnel and Guidance A ssocia tion 
Edwin L. Herr, Pennsylvania State University 

AIM /ARM 

Joel E. Magisos, AIM/ARM, Center for Vocational Education, The Ohio 
State University 

Project Baseline - ^ 

Arthur M. Lee, Director 




Appendix B 

American Vocational Education Research Association 
J erome Moss, Jr., University of Minnesota 

American Vocational Association, Inc, 

Mairy Ellis, Technical Education Research Center 

U.S. Chamber of Commerce 

Thomas P. Walsh, Associate Director, Education and Welfare 
A FLIC to 

John A. Sessions, Assistant Director for Education 

American Association of Community and Junior Colleges 
Sandy Drake, Director, Research & Data Gathering Department 
Network Council 

Joseph Kelly, Director, Northeast Curriculum Coordinating Center 

Council for Educational Development and Research, Inc. 

John K. Coster, Center for Occupational Education (North Carolina) 

ERIC Clearinghouse in Career Education . ^ 

David Tiedeman, Northern Illinois University 

Richard Erickson, Northern Illinois University 



All witnesses were requested to supply the Committee mih written 
testimony prior to the hearings. The Committee and its staff developed a 
set of preliminary questions to help participants organize their written . 
comments. The questions solicited information on the rhanagement 
problems of vocational education r&d, issues requiring additional Re- 
search, the role or functions of r&d, and evidence ohhe impact of r&d. 
Nearly all witnesses supplied the Committeewith written responses. 

Another major Committee effort was a survey conducted in September 
1975. Committee members identified 16 people whose research was relat- 
ed to vocational education and who cquld cite significant contributions 
made by vocational education r&d over the last decade;. 

Garry R. Bice, Tennessee Research Coordinating Unit 

Robert S. Campbell, Center for Vocational Education, Ohio State University 



124 



Appendix B > Jjy 

John Coster, Center for Occupational Education, North Carolina State 
University 

Mary Ellis, President, American Vocational Association 

Edwin L. Herr, Pennsylvania State University 

Robert Hoppock, New York University 

Ray E. Hosford, University of California, Santa Barbara 

G. Brian Jones, American Institutes for Research / 

Jacob J. Kaufman, Pennsylvania State University 

^ary B. Kievit, Rutgers University 

Edward Morrison, Ohio State University 

Jerome Moss, University of Minnesota 

Herbert Parnes, Ohio State University 
Merle E. Strong, University of Wisconsin 
Grant Venn, Georgia State University 

Louise Vetter, Center fcr Vocational Education, Ohio State University 

A letter posing questions was mailed to these people, followed by a 
telephone interview conducted by the Committee's staff. 

The Committee informally interviewed mdiny other people who are 
directly or indirectly involved in vocational education r&d . Program 
officers from the Office of Education were coni?ulted frequently. The staff • 
attended meetings of the National Network i;or Curriculum Coordina- 
tion in Vocational and Tech nica| /Education and a national meeting of 
the RCU directors. In additioiri, the Committee invited Gany Bice of the 
Tennessee RCU and Eugene Lehrmann, Wisconsin's Director of Voca- 
tional Education, to participate at some of its meetings as resource indi- 
viduals. Finally, the Committee conducted.two small surveys of the State 
Advisory Councils on Vocational Education in order to determine the 
extent of their involvement in r&d and to describe their relationships 
with the state rcus. In all> more than 50 people were interviewed and 
many more were contacted by rnail. 
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Priority Area and 
Project Type 
Section 4(c), 
Parts C, D, and I 



TABLE CI Sample of 149 Research Awards by Priority Areas, Section 4(c), 
Fiscal 1965-1969 



Annual Percentage of All Dollar Awards 



Priority Area 


1965 


1966 


1967 


1968 


1969 


Curriculum Development 


32.0 


30.2 


4.0 


53.9 


16.4 


Program Organization'and 












Administration (RCUs) 


21.6 


32.8 


36.3 


6.4 


4L6 


Career Development, Guidance 


3.1 


7.9 


«;9 


0 


1.0 


Application of Manpower Data to 












Occupational-Education 


0 


0 


0 


23.5 


11.4 


The Student and His Environment 


0 


0 


0 


l.l 


5.2 


Adult and Continuing Education 


13.1 


8.0 


0 


0 


d . 


Program Evaluation 


1.0 


3.7 


0.8 


9.4 


5.2 


Personnel Recruitment and 












Development 


11.2 


15.9 


10.9 


0 


0 


Miscellaneous* 


17.8 


1.1 


39.1 


5.4 


- 18.9 


TOTAL 


• 99.8 


99.6 


100.0 


99.7 


99.7 



•This category includes such priority areas as the personal and social significance of 
work, instructional facilities, planning techniques, and R&D centers. 
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TABLE C2 Research Awards by Priority Areas, Part C, Fiscal 1 971-1974 



Percentage of DoUar Awards 



Priority Area 


1971 


1972 


1973 


1974 


Curriculum Development 


43.2 


0 


0 


26.3 


State and Local Administration 


13.7 


0 


0 


0 


CsJ^i Education, Guidance 


27.8 


100.0 


100.0 


19.8 


Manpower Studies 


9.4 


0 


0 


26.7 


Disadvantaged, Handicapped, and Minorities 


0 . 


0 


0 


10.4 


The Student and His Environment 


5.8 


0 


0 


0 . 


Work Experience 


- 0 


0 


0 


16.8 


TOTAL 


99.9 


100.0 


100.0 


100.0 



TABLE C3 Number of Projects Reporting Federal Activity Components in 
Federal Share Part D Funds, Fiscal 1970-1973 





Number of Projects 


Percentage of 


Activities 


Reporting Participants 


50 Projects 


Elementary Activities 


49 


98 


Junior High Activities 


49 


^ 98 


Senior High Activities 






Placement 


37 


74 


Senior High Participating Teacher 


31 


62 


Work Experience Programs 


26 


52 


Job Entry Skill Training 


23 


46 


Intensive Guidance and Counseling 


22 


44 


Other Activities 


7 


14 



NOTE: This table prepared by Development Associates (1975, p. 45). 



TABLE C4 Percentage of Obligated Funds Attributed to Objectives, States' 
Share Part D, Fiscal 1 970-1 973 



Objective 


Year I 


Year II 


Year III 


Occupational Familiarization 


23 


24 


73 


Work Experience 


14 


14 


1^ 


Guidance and Placement 


13 


12 


13 


Curriculum Improvement 


25 


26 


25 


Exchange of Personnel 


7 


7 


6 


Young Workers : 


4 


4 


4 


Vocational Education Careers 


7 


8 


9 


Other 


6 


6 


6 



NOTE: This table prepared by Development Associates (1975» p. 146). 
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TABLE C6 Research Awards by Project Type, Section 4(c), 
Fiscal 1965-1969 



Annual Percentage of All Dollar Awards 



Type f 


1965 


1966 


1967 


1968 


1969 


Total 


Experimental, 














Developmental, or Pilot 


36.8 


51.1 


4.1 


62.9 


19.1 - 


37.4 


Research 


27.9 


31.6 


16.8 


23.4 


48.5 


31.9 


Training. 


9.1 


8.1 


14.2 


9.6 


4.2 


8.4 


Research Coordinating Unit 


17.8 


9.2 


35.7 


4.0 


12.6 


14.1 


National R&D Center 


8.4 


0 


29.2 


0 


15.6 


8.3 


TOTAL 


100.0 


100.0 


100.0 ' 


99.9 


100.0 


100.1 



TABLE C7 Research Awards by Project Type, Federal Fart C, 
Fiscal 197 M 974 . ... 



Type 


Annual Percentage o 


f All Dollar Awards 






1971 


1972* 


1973* 


1974 


Total 


Experimental, Devel- 












opmental, or Pilot 


68. 1 


100.0 


100.0 


30.8 


74.4 


Research 


11.9 


0 


0 


67.9 


17.2 


Training 


0.8 


0 


0 


1.2 


0.5 


Curriculum Centers 


19.2 


0 


0 


0 


7'.8 


TOTAL 


100.0 


100.0 


100.0 


99.9 


99.9 



*A11 projects funded in fiscal 1972 and 1973 were career education models. 
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TABLE C9 Classification of Projects by Oceupational Cluster, Part I 
Tiscal 1972-1974 



Total 



Cluster 1972 



1973 1974 Number . Percentage 



Ag: ibusiness and Natural 

Resources 
BuMiiejsand Office 
Ccmmunications and Media 
C^.'^;struction 

Consumer and Homemaking 

Enviromnent 

Fir.r: Arts and Humanities 

Manufacturing 

Marine Science 

Marl.iiting and Distribution 

Personal Services 

Puc.<ic Service 

Ret -cation and Hospitality 

Transportation 

Combination 

TOTAL 



1 . 


0 


0 


1 


1.7 


1 


3 


2 


6 


10.2 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


' 2 


■ 0 


1 


3 


5.1 


0 


-r 


0 


0 


0 


0 




1 


2 


3.4 


0 


1 


0 


r 


1.7 


3 


1 


0 


4 


6.8 


0 


0 


1 


1 


1.7 


1 


1 


0 


2 ^ 


3.4 


0 


0 


2 


2 


3.4 


0 


0 


0 


0 


. 0 


0 


0 


1 


1 


• 1.'7 


0 


0 


0 


0 


0 


20 


6 


10 


36 


61.0 


28 


13 


18 


59 


100.1 
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